Managing devices having additive displays

ABSTRACT

One or more techniques for managing virtual objects between one or more displays are described. In accordance with some embodiments, exemplary techniques for displaying a virtual object are described.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT/US2020/048842, entitled“MANAGING DEVICES HAVING ADDITIVE DISPLAYS,” filed Aug. 31, 2020, whichclaims priority to U.S. Provisional Patent Application Ser. No.63/067,461, entitled “MANAGING DEVICES HAVING ADDITIVE DISPLAYS,” filedon Aug. 19, 2020, and U.S. Provisional Patent Application Ser. No.62/907,207, entitled “TECHNIQUES FOR MANAGING DEVICES HAVING ADDITIVEDISPLAYS FOR USE IN A COMPUTER-GENERATED REALITY ENVIRONMENT,” filed onSep. 27, 2019. The contents of the aforementioned applications arehereby incorporated by reference in their entireties.

FIELD

The present disclosure generally relates to computer-generated realityenvironments, and, more specifically, to methods and techniques formanaging devices having additive displays for use in computer-generatedreality environments.

BACKGROUND

Users often use wearable devices, such as head-mounted display (HMD)devices, to interact with computer-generated reality environments. Theuse of a wearable device is often limited by the amount of the wearabledevice's battery life. Thus, one or more techniques are needed to managepower usage to conserve battery life in wearable devices.

BRIEF SUMMARY

In accordance with some embodiments, a method is performed at a systemhaving one or more processors, memory, one or more image sensors, and ahead-mounted display (HMD) device, the HMD device including a primarydisplay that extends across a field-of-view and has a first resolution,and a secondary display that is physically and electronically coupled tothe primary display and has a second resolution that is lower than thefirst resolution. The method includes: displaying a first portion of avirtual object via the secondary display; and displaying a secondportion of the virtual object via the primary display, whereindisplaying the second portion of the virtual object via the primarydisplay includes: in accordance with a determination that the secondportion of the virtual object is within a predefined distance from anedge of the primary display, applying a visual effect to the secondportion of the virtual object.

In accordance with some embodiments, a head-mounted display (HMD) deviceincludes one or more image sensors, a primary display that extendsacross a field-of-view and has a first resolution, and a secondarydisplay that is physically and electronically coupled to the primarydisplay and has a second resolution that is lower than the firstresolution. The HMD device also includes one or more processors; andmemory storing one or more programs configured to be executed by the oneor more processors. The one or more programs include instructions for:displaying a first portion of a virtual object via the secondarydisplay; and displaying a second portion of the virtual object via theprimary display, wherein displaying the second portion of the virtualobject via the primary display includes: in accordance with adetermination that the second portion of the virtual object is within apredefined distance from an edge of the primary display, applying avisual effect to the second portion of the virtual object.

In accordance with some embodiments, a non-transitory computer-readablestorage medium storing one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, and a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution. The one or moreprograms include instructions for: displaying a first portion of avirtual object via the secondary display; and displaying a secondportion of the virtual object via the primary display, whereindisplaying the second portion of the virtual object via the primarydisplay includes: in accordance with a determination that the secondportion of the virtual object is within a predefined distance from anedge of the primary display, applying a visual effect to the secondportion of the virtual object.

In accordance with some embodiments, a transitory computer-readablestorage medium storing one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, and a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution. The one or moreprograms include instructions for: displaying a first portion of avirtual object via the secondary display; and displaying a secondportion of the virtual object via the primary display, whereindisplaying the second portion of the virtual object via the primarydisplay includes: in accordance with a determination that the secondportion of the virtual object is within a predefined distance from anedge of the primary display, applying a visual effect to the secondportion of the virtual object.

In accordance with some embodiments, a system includes one or moreprocessors. The system also includes means for displaying a firstportion of a virtual object via a secondary display; and means fordisplaying a second portion of the virtual object via a primary display,wherein displaying the second portion of the virtual object via theprimary display includes: in accordance with a determination that thesecond portion of the virtual object is within a predefined distancefrom an edge of the primary display, applying a visual effect to thesecond portion of the virtual object.

In accordance with some embodiments, a method is performed at a systemhaving one or more processors, memory, one or more image sensors, and ahead-mounted display (HMD) device, the HMD device including a primarydisplay that extends across a field-of-view and has a first resolution,a secondary display that is physically and electronically coupled to theprimary display and has a second resolution that is lower than the firstresolution, and a tertiary display. The method includes receivingdirectional information corresponding to a location of an object that isoutside a field-of-view of the primary display of the HMD device and thesecondary display of the HMD device. The method also includes, inresponse to receiving the directional information corresponding to thelocation of the object that is outside the field-of-view of the primarydisplay and the secondary display, in accordance with a determinationthat first criteria are satisfied, wherein the first criteria includes acriterion that is met when directional information for the object ispermitted to be displayed via the secondary display, displaying, via thesecondary display, a first representation of directional information;and in accordance with a determination that second criteria aresatisfied, wherein the second criteria includes a criterion that is metwhen directional information for the object is permitted to be displayedvia the tertiary display, displaying, via the tertiary display, a secondrepresentation of directional information, wherein the firstrepresentation is a virtual object within a CGR environment, and whereinthe second representation is not a virtual object within the CGRenvironment.

In accordance with some embodiments, a head-mounted display (HMD) deviceincludes one or more image sensors, a primary display that extendsacross a field-of-view and has a first resolution, a secondary displaythat is physically and electronically coupled to the primary display andhas a second resolution that is lower than the first resolution, and atertiary display. The HMD device also includes one or more processors;and memory storing one or more programs configured to be executed by theone or more processors. The one or more programs include instructionsfor receiving directional information corresponding to a location of anobject that is outside a field-of-view of the primary display of the HMDdevice and the secondary display of the HMD device. The one or moreprograms also include instructions for, in response to receiving thedirectional information corresponding to the location of the object thatis outside the field-of-view of the primary display and the secondarydisplay, in accordance with a determination that first criteria aresatisfied, wherein the first criteria includes a criterion that is metwhen directional information for the object is permitted to be displayedvia the secondary display, displaying, via the secondary display, afirst representation of directional information; and in accordance witha determination that second criteria are satisfied, wherein the secondcriteria includes a criterion that is met when directional informationfor the object is permitted to be displayed via the tertiary display,displaying, via the tertiary display, a second representation ofdirectional information, wherein the first representation is a virtualobject within a CGR environment, and wherein the second representationis not a virtual object within the CGR environment.

In accordance with some embodiments, a non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution, and a tertiarydisplay. The one or more programs include instructions for receivingdirectional information corresponding to a location of an object that isoutside a field-of-view of the primary display of the HMD device and thesecondary display of the HMD device. The one or more programs alsoinclude instructions for, in response to receiving the directionalinformation corresponding to the location of the object that is outsidethe field-of-view of the primary display and the secondary display, inaccordance with a determination that first criteria are satisfied,wherein the first criteria includes a criterion that is met whendirectional information for the object is permitted to be displayed viathe secondary display, displaying, via the secondary display, a firstrepresentation of directional information; and in accordance with adetermination that second criteria are satisfied, wherein the secondcriteria includes a criterion that is met when directional informationfor the object is permitted to be displayed via the tertiary display,displaying, via the tertiary display, a second representation ofdirectional information, wherein the first representation is a virtualobject within a CGR environment, and wherein the second representationis not a virtual object within the CGR environment.

In accordance with some embodiments, a transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution, and a tertiarydisplay. The one or more programs include instructions for receivingdirectional information corresponding to a location of an object that isoutside a field-of-view of the primary display of the HMD device and thesecondary display of the HMD device. The one or more programs alsoinclude instructions for, in response to receiving the directionalinformation corresponding to the location of the object that is outsidethe field-of-view of the primary display and the secondary display, inaccordance with a determination that first criteria are satisfied,wherein the first criteria includes a criterion that is met whendirectional information for the object is permitted to be displayed viathe secondary display, displaying, via the secondary display, a firstrepresentation of directional information; and in accordance with adetermination that second criteria are satisfied, wherein the secondcriteria includes a criterion that is met when directional informationfor the object is permitted to be displayed via the tertiary display,displaying, via the tertiary display, a second representation ofdirectional information, wherein the first representation is a virtualobject within a CGR environment, and wherein the second representationis not a virtual object within the CGR environment.

In accordance with some embodiments, a system includes one or moreprocessors. The system also includes means for receiving directionalinformation corresponding to a location of an object that is outside afield-of-view of a primary display of a head-mounted display (HMD)device and a secondary display of the HMD device. The system furtherincludes means, responsive to receiving the directional informationcorresponding to the location of the object that is outside thefield-of-view of the primary display and the secondary display, for: inaccordance with a determination that first criteria are satisfied,wherein the first criteria includes a criterion that is met whendirectional information for the object is permitted to be displayed viathe secondary display, displaying, via a secondary display, a firstrepresentation of directional information; and in accordance with adetermination that second criteria are satisfied, wherein the secondcriteria includes a criterion that is met when directional informationfor the object is permitted to be displayed via the tertiary display,displaying, via a tertiary display, a second representation ofdirectional information, wherein the first representation is a virtualobject within a CGR environment, and wherein the second representationis not a virtual object within the CGR environment.

In accordance with some embodiments, a method is performed at a systemhaving one or more processors, memory, one or more image sensors, and ahead-mounted display (HMD) device, the HMD device including a primarydisplay that extends across a field-of-view and has a first resolution,and a secondary display that is physically and electronically coupled tothe primary display and has a second resolution that is lower than thefirst resolution. The method includes: detecting an object in acomputer-generated reality (CGR) environment at a first location; and inresponse to detecting the object in the CGR environment at the firstlocation, in accordance with a determination that the first location iswithin a first predetermined distance outside of the field-of-view ofthe primary display of the HMD device, displaying, via the secondarydisplay of the HMD device, a first modified representation of the objectthat is visually distinguished from a first actual representation of theobject.

In accordance with some embodiments, a head-mounted display (HMD) deviceincludes one or more image sensors, a primary display that extendsacross a field-of-view and has a first resolution, and a secondarydisplay that is physically and electronically coupled to the primarydisplay and has a second resolution that is lower than the firstresolution. The HMD device also includes one or more processors; andmemory storing one or more programs configured to be executed by the oneor more processors. The one or more programs include instructions for:detecting an object in a computer-generated reality (CGR) environment ata first location; and in response to detecting the object in the CGRenvironment at the first location, in accordance with a determinationthat the first location is within a first predetermined distance outsideof the field-of-view of the primary display of the HMD device,displaying, via the secondary display of the HMD device, a firstmodified representation of the object that is visually distinguishedfrom a first actual representation of the object.

In accordance with some embodiments, a non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, and a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution. The one or moreprograms include instructions for: detecting an object in acomputer-generated reality (CGR) environment at a first location; and inresponse to detecting the object in the CGR environment at the firstlocation, in accordance with a determination that the first location iswithin a first predetermined distance outside of the field-of-view ofthe primary display of the HMD device, displaying, via the secondarydisplay of the HMD device, a first modified representation of the objectthat is visually distinguished from a first actual representation of theobject.

In accordance with some embodiments, a transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, and a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution. The one or moreprograms include instructions for: detecting an object in acomputer-generated reality (CGR) environment at a first location; and inresponse to detecting the object in the CGR environment at the firstlocation, in accordance with a determination that the first location iswithin a first predetermined distance outside of the field-of-view ofthe primary display of the HMD device, displaying, via the secondarydisplay of the HMD device, a first modified representation of the objectthat is visually distinguished from a first actual representation of theobject.

In accordance with some embodiments, a system includes one or moreprocessors. The system also includes means for detecting an object in acomputer-generated reality (CGR) environment at a first location. Thesystem further includes means responsive to, detecting the object in theCGR environment at the first location, for, in accordance with adetermination that the first location is within a first predetermineddistance outside of the field-of-view of a primary display of the HMDdevice, displaying, via a secondary display of the HMD device, a firstmodified representation of the object that is visually distinguishedfrom a first actual representation of the object.

In accordance with some embodiments, a method is performed at a systemhaving one or more processors, memory, one or more image sensors, and ahead-mounted display (HMD) device, the HMD device including a primarydisplay that extends across a field-of-view and has a first resolution,a secondary display that is physically and electronically coupled to theprimary display and has a second resolution that is lower than the firstresolution, and a tertiary display. The method includes receivingdirectional information corresponding to a location of an object that isoutside a field-of-view of the primary display of the HMD device and thesecondary display of the HMD device. In response to receivinginformation corresponding to the change status of the process, themethod also includes: in accordance with a determination that a firstcriterion is satisfied, displaying, via the secondary display of the HMDdevice, a first representation corresponding to the status of theprocess; and in accordance with a determination that a second criterionis satisfied, displaying, via the tertiary display of the HMD device, asecond representation corresponding to the status of the process,wherein the second representation is different from the firstrepresentation.

In accordance with some embodiments, a head-mounted display (HMD) deviceincludes one or more image sensors, a primary display that extendsacross a field-of-view and has a first resolution, a secondary displaythat is physically and electronically coupled to the primary display andhas a second resolution that is lower than the first resolution, and atertiary display. The HMD device also includes one or more processors;and memory storing one or more programs configured to be executed by theone or more processors. The one or more programs include instructionsfor receiving information corresponding to a change in a status of aprocess. In response to receiving information corresponding to thechange status of the process, the one or more programs also include: inaccordance with a determination that a first criterion is satisfied,displaying, via the secondary display of the HMD device, a firstrepresentation corresponding to the status of the process; and inaccordance with a determination that a second criterion is satisfied,displaying, via the tertiary display of the HMD device, a secondrepresentation corresponding to the status of the process, wherein thesecond representation is different from the first representation.

In accordance with some embodiments, a non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution, and a tertiarydisplay. The one or more programs include instructions for receivinginformation corresponding to a change in a status of a process. Inresponse to receiving information corresponding to the change status ofthe process, the one or more programs also include: in accordance with adetermination that a first criterion is satisfied, displaying, via thesecondary display of the HMD device, a first representationcorresponding to the status of the process; and in accordance with adetermination that a second criterion is satisfied, displaying, via thetertiary display of the HMD device, a second representationcorresponding to the status of the process, wherein the secondrepresentation is different from the first representation.

In accordance with some embodiments, a transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moresensors, a primary display that extends across a field-of-view and has afirst resolution, a secondary display that is physically andelectronically coupled to the primary display and has a secondresolution that is lower than the first resolution, and a tertiarydisplay. The one or more programs include instructions for receivinginformation corresponding to a change in a status of a process. Inresponse to receiving information corresponding to the change status ofthe process, the one or more programs also include: in accordance with adetermination that a first criterion is satisfied, displaying, via thesecondary display of the HMD device, a first representationcorresponding to the status of the process; and in accordance with adetermination that a second criterion is satisfied, displaying, via thetertiary display of the HMD device, a second representationcorresponding to the status of the process, wherein the secondrepresentation is different from the first representation.

In accordance with some embodiments, a system includes one or moreprocessors. The system also includes means for receiving directionalinformation corresponding to a location of an object that is outside afield-of-view of a primary display of a head-mounted display (HMD)device and a secondary display of the HMD device. The system furtherincludes means, responsive to receiving information corresponding to thechange status of the process, for: in accordance with a determinationthat a first criterion is satisfied, displaying, via the secondarydisplay of the HMD device, a first representation corresponding to thestatus of the process; and in accordance with a determination that asecond criterion is satisfied, displaying, via the tertiary display ofthe HMD device, a second representation corresponding to the status ofthe process, wherein the second representation is different from thefirst representation.

Executable instructions for performing these functions are, optionally,included in a non-transitory computer-readable storage medium or othercomputer program product configured for execution by one or moreprocessors. Executable instructions for performing these functions are,optionally, included in a transitory computer-readable storage medium orother computer program product configured for execution by one or moreprocessors.

BRIEF DESCRIPTION OF THE FIGURES

In the following description, reference is made to the accompanyingfigures which form a part thereof, and which illustrate several examplesof the present disclosure. It is understood that other examples may beutilized and structural and operational changes may be made withoutdeparting from the scope of the present disclosure. The use of the samereference symbols in different figures indicates similar or identicalitems.

FIGS. 1A-1B depict exemplary systems for use in variouscomputer-generated reality technologies.

FIGS. 2A-2B depict an exemplary system that includes a device that hasadditive displays for use in various computer-generated realityenvironments.

FIG. 3 depicts an exemplary virtual object in accordance with someembodiments.

FIG. 4 depicts an exemplary technique for displaying a virtual object inaccordance with some embodiments.

FIG. 5 depicts exemplary regions on a device that has additive displaysfor displaying a virtual object in accordance with some embodiments.

FIGS. 6A-6C depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIG. 7 is a flow diagram illustrating a method for displaying a virtualobject in accordance with some embodiments.

FIGS. 8A-8B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 9A-9B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 10A-10B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 11A-11B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 12A-12B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 13A-13B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIGS. 14A-14B depict an exemplary technique for displaying a virtualobject in accordance with some embodiments.

FIG. 15 is a flow diagram illustrating a method for transitioningobjects between displays based on directional information in accordancewith some embodiments.

FIG. 16 is a flow diagram illustrating a method for displaying amodified representation of an object in accordance with someembodiments.

FIGS. 17A-17B depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 18 depicts an exemplary technique for managing one or more displaysbased on data associated with one or more processes in accordance withsome embodiments.

FIGS. 19A-19B depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIGS. 20A-20D depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIGS. 21A-21D depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIGS. 22A-22B depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIGS. 23A-23B depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIGS. 24A-24D depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 25 is a flow diagram illustrating a method for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments.

DESCRIPTION

Various examples of electronic systems and techniques for using suchsystems in relation to various computer-generated reality technologiesare described.

A physical environment (or real environment) refers to a physical worldthat people can sense and/or interact with without aid of electronicsystems. Physical environments, such as a physical park, includephysical articles (or physical objects or real objects), such asphysical trees, physical buildings, and physical people. People candirectly sense and/or interact with the physical environment, such asthrough sight, touch, hearing, taste, and smell.

In contrast, a computer-generated reality (CGR) environment refers to awholly or partially simulated environment that people sense and/orinteract with via an electronic system. In CGR, a subset of a person'sphysical motions, or representations thereof, are tracked, and, inresponse, one or more characteristics of one or more virtual objectssimulated in the CGR environment are adjusted in a manner that comportswith at least one law of physics. For example, a CGR system may detect aperson's head turning and, in response, adjust graphical content and anacoustic field presented to the person in a manner similar to how suchviews and sounds would change in a physical environment. In somesituations (e.g., for accessibility reasons), adjustments tocharacteristic(s) of virtual object(s) in a CGR environment may be madein response to representations of physical motions (e.g., vocalcommands).

A person may sense and/or interact with a CGR object using any one oftheir senses, including sight, sound, touch, taste, and smell. Forexample, a person may sense and/or interact with audio objects thatcreate a 3D or spatial audio environment that provides the perception ofpoint audio sources in 3D space. In another example, audio objects mayenable audio transparency, which selectively incorporates ambient soundsfrom the physical environment with or without computer-generated audio.In some CGR environments, a person may sense and/or interact only withaudio objects.

Examples of CGR include virtual reality and mixed reality.

A virtual reality (VR) environment (or virtual environment) refers to asimulated environment that is designed to be based entirely oncomputer-generated sensory inputs for one or more senses. A VRenvironment comprises a plurality of virtual objects with which a personmay sense and/or interact. For example, computer-generated imagery oftrees, buildings, and avatars representing people are examples ofvirtual objects. A person may sense and/or interact with virtual objectsin the VR environment through a simulation of the person's presencewithin the computer-generated environment, and/or through a simulationof a subset of the person's physical movements within thecomputer-generated environment.

In contrast to a VR environment, which is designed to be based entirelyon computer-generated sensory inputs, a mixed reality (MR) environmentrefers to a simulated environment that is designed to incorporatesensory inputs from the physical environment, or a representationthereof, in addition to including computer-generated sensory inputs(e.g., virtual objects). On a virtuality continuum, an MR environment isanywhere between, but not including, a wholly physical environment atone end and a VR environment at the other end.

In some MR environments, computer-generated sensory inputs may respondto changes in sensory inputs from the physical environment. Also, someelectronic systems for presenting an MR environment may track locationand/or orientation with respect to the physical environment to enablevirtual objects to interact with real objects (that is, physicalarticles from the physical environment or representations thereof). Forexample, a system may account for movements so that a virtual treeappears stationary with respect to the physical ground.

Examples of MR include augmented reality and augmented virtuality.

An augmented reality (AR) environment refers to a simulated environmentin which one or more virtual objects are superimposed over a physicalenvironment, or a representation thereof. For example, an electronicsystem for presenting an AR environment may have a transparent ortranslucent display through which a person may directly view thephysical environment. The system may be configured to present virtualobjects on the transparent or translucent display, so that a person,using the system, perceives the virtual objects superimposed over thephysical environment. Alternatively, a system may have an opaque displayand one or more imaging sensors that capture images or video of thephysical environment, which are representations of the physicalenvironment. The system composites the images or video with virtualobjects, and presents the composition on the opaque display. A person,using the system, indirectly views the physical environment by way ofthe images or video of the physical environment, and perceives thevirtual objects superimposed over the physical environment. As usedherein, a video of the physical environment shown on an opaque displayis called “pass-through video,” meaning a system uses one or more imagesensor(s) to capture images of the physical environment, and uses thoseimages in presenting the AR environment on the opaque display. Furtheralternatively, a system may have a projection system that projectsvirtual objects into the physical environment, for example, as ahologram or on a physical surface, so that a person, using the system,perceives the virtual objects superimposed over the physicalenvironment.

An AR environment also refers to a simulated environment in which arepresentation of a physical environment is transformed bycomputer-generated sensory information. For example, in providingpass-through video, a system may transform one or more sensor images toimpose a select perspective (e.g., viewpoint) different than theperspective captured by the imaging sensors. As another example, arepresentation of a physical environment may be transformed bygraphically modifying (e.g., enlarging) portions thereof, such that themodified portion may be representative but not photorealistic versionsof the originally captured images. As a further example, arepresentation of a physical environment may be transformed bygraphically eliminating or obfuscating portions thereof.

An augmented virtuality (AV) environment refers to a simulatedenvironment in which a virtual or computer-generated environmentincorporates one or more sensory inputs from the physical environment.The sensory inputs may be representations of one or more characteristicsof the physical environment. For example, an AV park may have virtualtrees and virtual buildings, but people with faces photorealisticallyreproduced from images taken of physical people. As another example, avirtual object may adopt a shape or color of a physical article imagedby one or more imaging sensors. As a further example, a virtual objectmay adopt shadows consistent with the position of the sun in thephysical environment.

There are many different types of electronic systems that enable aperson to sense and/or interact with various CGR environments. Examplesinclude head-mounted systems, projection-based systems, heads-updisplays (HUDs), vehicle windshields having integrated displaycapability, windows having integrated display capability, displaysformed as lenses designed to be placed on a person's eyes (e.g., similarto contact lenses), headphones/earphones, speaker arrays, input systems(e.g., wearable or handheld controllers with or without hapticfeedback), smartphones, tablets, and desktop/laptop computers. Ahead-mounted system may have one or more speaker(s) and an integratedopaque display. Alternatively, a head-mounted system may be configuredto accept an external opaque display (e.g., a smartphone). Thehead-mounted system may incorporate one or more imaging sensors tocapture images or video of the physical environment, and/or one or moremicrophones to capture audio of the physical environment. Rather than anopaque display, a head-mounted system may have a transparent ortranslucent display. The transparent or translucent display may have amedium through which light representative of images is directed to aperson's eyes. The display may utilize digital light projection, OLEDs,LEDs, uLEDs, liquid crystal on silicon, laser scanning light source, orany combination of these technologies. The medium may be an opticalwaveguide, a hologram medium, an optical combiner, an optical reflector,or any combination thereof. In one example, the transparent ortranslucent display may be configured to become opaque selectively.Projection-based systems may employ retinal projection technology thatprojects graphical images onto a person's retina. Projection systemsalso may be configured to project virtual objects into the physicalenvironment, for example, as a hologram or on a physical surface.

FIG. 1A and FIG. 1B depict exemplary system 100 for use in variouscomputer-generated reality technologies.

In some examples, as illustrated in FIG. 1A, system 100 includes device100 a. Device 100 a includes various components, such as processor(s)102, RF circuitry(ies) 104, memory(ies) 106, image sensor(s) 108,orientation sensor(s) 110, microphone(s) 112, location sensor(s) 116,speaker(s) 118, display(s) 120, and touch-sensitive surface(s) 122.These components optionally communicate over communication bus(es) 150of device 100 a.

In some examples, elements of system 100 are implemented in a basestation device (e.g., a computing device, such as a remote server,mobile device, or laptop) and other elements of the system 100 areimplemented in a head-mounted display (HMD) device designed to be wornby the user, where the HMD device is in communication with the basestation device. In some examples, device 100 a is implemented in a basestation device or an HMD device.

As illustrated in FIG. 1B, in some examples, system 100 includes two (ormore) devices in communication, such as through a wired connection or awireless connection. First device 100 b (e.g., a base station device)includes processor(s) 102, RF circuitry(ies) 104, and memory(ies) 106.These components optionally communicate over communication bus(es) 150of device 100 b. Second device 100 c (e.g., a head-mounted device)includes various components, such as processor(s) 102, RF circuitry(ies)104, memory(ies) 106, image sensor(s) 108, orientation sensor(s) 110,microphone(s) 112, location sensor(s) 116, speaker(s) 118, display(s)120, and touch-sensitive surface(s) 122. These components optionallycommunicate over communication bus(es) 150 of device 100 c.

In some examples, system 100 is a mobile device. In some examples,system 100 is an HMD device. In some examples, system 100 is a wearableHUD device.

System 100 includes processor(s) 102 and memory(ies) 106. Processor(s)102 include one or more general processors, one or more graphicsprocessors, and/or one or more digital signal processors. In someexamples, memory(ies) 106 are one or more non-transitorycomputer-readable storage mediums (e.g., flash memory, random accessmemory) that store computer-readable instructions configured to beexecuted by processor(s) 102 to perform the techniques described below.

System 100 includes RF circuitry(ies) 104. RF circuitry(ies) 104optionally include circuitry for communicating with electronic devices,networks, such as the Internet, intranets, and/or a wireless network,such as cellular networks and wireless local area networks (LANs). RFcircuitry(ies) 104 optionally includes circuitry for communicating usingnear-field communication and/or short-range communication, such asBluetooth®.

System 100 includes display(s) 120. In some examples, display(s) 120include a first display (e.g., a left eye display panel) and a seconddisplay (e.g., a right eye display panel), each display for displayingimages to a respective eye of the user. Corresponding images aresimultaneously displayed on the first display and the second display.Optionally, the corresponding images include the same virtual objectsand/or representations of the same physical objects from differentviewpoints, resulting in a parallax effect that provides a user with theillusion of depth of the objects on the displays. In some examples, thefirst display can include a plurality of other displays (e.g.,sub-displays), such as a primary display and a secondary display. Insome embodiments, the primary display has a different resolution thanthe secondary display when system 100 is operating. In some examples,display(s) 120 include a single display (e.g., first display or seconddisplay). In some examples, the single display includes a plurality ofother displays (e.g., sub-displays), such as primary display andsecondary display. Corresponding images are simultaneously displayed ona first area and a second area of the single display for each eye of theuser. Optionally, the corresponding images include the same virtualobjects and/or representations of the same physical objects fromdifferent viewpoints, resulting in a parallax effect that provides auser with the illusion of depth of the objects on the single display.

In some examples, system 100 includes touch-sensitive surface(s) 122 forreceiving user inputs, such as tap inputs and swipe inputs. In someexamples, display(s) 120 and touch-sensitive surface(s) 122 formtouch-sensitive display(s).

System 100 includes image sensor(s) 108. Image sensor(s) 108 optionallyinclude one or more visible light image sensor(s), such as chargedcoupled device (CCD) sensors, and/or complementarymetal-oxide-semiconductor (CMOS) sensors operable to obtain images ofphysical objects from the real environment. Image sensor(s) alsooptionally include one or more infrared (IR) sensor(s), such as apassive IR sensor or an active IR sensor, for detecting infrared lightfrom the real environment. For example, an active IR sensor includes anIR emitter, such as an IR dot emitter, for emitting infrared light intothe real environment. Image sensor(s) 108 also optionally include one ormore event camera(s) configured to capture movement of physical objectsin the real environment. Image sensor(s) 108 also optionally include oneor more depth sensor(s) configured to detect the distance of physicalobjects from system 100. In some examples, system 100 uses CCD sensors,event cameras, and depth sensors in combination to detect the physicalenvironment around system 100. In some examples, image sensor(s) 108include a first image sensor and a second image sensor. The first imagesensor and the second image sensor are optionally configured to captureimages of physical objects in the real environment from two distinctperspectives. In some examples, system 100 uses image sensor(s) 108 toreceive user inputs, such as hand gestures. In some examples, system 100uses image sensor(s) 108 to detect the position and orientation ofsystem 100 and/or display(s) 120 in the real environment. For example,system 100 uses image sensor(s) 108 to track the position andorientation of display(s) 120 relative to one or more fixed objects inthe real environment.

In some examples, system 100 includes microphone(s) 112. System 100 usesmicrophone(s) 112 to detect sound from the user and/or the realenvironment of the user. In some examples, microphone(s) 112 includes anarray of microphones (including a plurality of microphones) thatoptionally operate in tandem, such as to identify ambient noise or tolocate the source of sound in the space of the real environment.

System 100 includes orientation sensor(s) 110 for detecting orientationand/or movement of system 100 and/or display(s) 120. For example, system100 uses orientation sensor(s) 110 to track changes in the positionand/or orientation of system 100 and/or display(s) 120, such as withrespect to physical objects in the real environment. Orientationsensor(s) 110 optionally include one or more gyroscopes and/or one ormore accelerometers.

FIG. 2A illustrates device 202. In some embodiments, device 202 caninclude one or more components of system 100, such as one or moreprocessors (e.g., processor(s) 102), memories (e.g., memory(ies) 106),camera sensor(s) (e.g., image sensor(s) 108) or motion sensor(s) (e.g.,orientation sensor(s) 110). For example, device 202 can be an embodimentof device 100 a of system 100 depicted in FIG. 1A. Alternatively, insome embodiments, device 202 can be an embodiment of second device 100 cdepicted in FIG. 1B.

As illustrated in FIG. 2A, device 202 is a wearable device that includesa first display (e.g., left eye display panel of device 202) and asecond display (e.g., right eye display panel of device 202). The firstdisplay and second display each include a primary display 204 that issurrounded by a secondary display 206. As illustrated in FIG. 2A,secondary display 206 circumscribes at least a portion of primarydisplay 204. In some embodiments, secondary display 206 extends acrossthe entirety of the first or second display. In some embodiments,primary display 204 overlaps only a portion (e.g., center) of secondarydisplay 206.

In some embodiments, primary display 204 and secondary display 206 aretwo different types of displays. In some embodiments, primary display204 is a wave guide display. In some embodiments, secondary display 206is an organic light-emitting diode display.

In FIG. 2A, primary display 204 is physically coupled (e.g., connected,attached) to secondary display 206 as well as electronically coupled(e.g., connected, attached) to secondary display 206. In someembodiments, one or more regions or additional displays may be disposedbetween primary display 204 and secondary display 206.

In FIG. 2A, primary display 204 and secondary display 206 are depictedas oval-shaped displays. It should be recognized, however, that primarydisplay 204 and secondary display 206 can have various shapes, such asrectangular shapes (e.g., rectangular prisms). Also, in someembodiments, the shape of primary display 204 can be different from theshape of secondary display 206. In some embodiments, when primarydisplay 204 and/or secondary display 206 are in the shape of arectangle, device 202 can apply a visual effect (e.g., blurring, fadingthe corners of the primary display and/or secondary display) to primarydisplay 204 and/or secondary display 206, such that primary display 204and/or secondary display 206 appear to be round.

As illustrated in FIG. 2A, edge 208 is defined between primary display204 and secondary display 206. In some embodiments, edge 208 istransparent and/or translucent. In some embodiments, edge 208 is an edgeof primary display 204 or an edge of secondary display 206. In someembodiments, edge 208 is a physical boundary between primary display 204and secondary display 206.

In some embodiments, primary display 204 can be operated at a maximumresolution that is higher than the maximum potential resolution ofsecondary display 206. Because displays that can operate at higherresolution typically require more energy, device 202 can use more energywhen displaying a virtual object (or CGR object) via primary display 204than when displaying the same object via secondary display 206. Thus,device 202 can reserve battery life based on a determination to displayone or more virtual objects in secondary display 206 in lieu ofdisplaying the one or more virtual objects in primary display 204.

As illustrated in FIG. 2A, device 202 includes frame 210 that ispositioned around secondary display 206. Frame 210 includes a tertiarydisplay, such as light-emitting diodes (LEDs). Alternatively, in someembodiments, frame 210 does not include a tertiary display.

As will be described below, in some embodiments, device 202 causes oneor more of the LEDs to emit light. In some embodiments, the LEDs do notsurround the inner portion of secondary 206 located near the nose of auser.

In FIG. 2B, device 202 is depicted being worn by user 250, such that thenose of user 250 is positioned under bridge 214 of device 202. As willbe described below, device 202 enables user 250 to view a CGRenvironment.

FIG. 2B depicts user 250 standing in a physical environment with theeyes of user 250 at a location such that pupil 216 of user 250 isaligned with a portion of primary display 204. Thereby, at FIG. 2B, thecentral vision or line of sight of user 250 is aligned with a portion ofthe field-of-view of primary display 204, which indicates that objectsdisplayed in primary display 204 are displayed in the user's centralvision.

Moreover, FIG. 2B depicts the peripheral vision of user 250 aligned witha portion of the field-of-view of secondary display 206, which indicatesthat objects displayed in secondary display 206 are displayed in theuser's peripheral vision. As will be described below, in someembodiments, when one or more LEDs are emitting light, the light isdisplayed in an exterior portion or edge of the peripheral vision ofuser 250. In some embodiments, one or more sensors of device 202 cantrack the movement of the user's eyes, including pupils 216. In someembodiments, device 202 uses the tracking of movement to adjust ormodify one or more of the techniques discussed below.

FIG. 3 depicts an exemplary virtual object in accordance with someembodiments. In particular, FIG. 3 depicts the virtual object as pencil302 shaded a solid color (e.g., black, blue). Pencil 302 has differentportions 302 a-302 c.

FIG. 4 depicts device 202 displaying a portion of pencil 302. Inparticular, in FIG. 4, pencil 302 is depicted falling from a positionthat is outside of the field-of-views of primary display 204 andsecondary display 206. Specifically, in FIG. 4, pencil 302 is depictedat a position where portions 302 a and 302 b of pencil 302 are in thefield-of-view of primary display 204 and portion 302 c of pencil 302 isin the field-of-view of secondary display 206, but the portion of pencil302 depicted in dashed lines in FIG. 4 is outside of the field-of-viewsof primary display 204 and secondary display 206.

As illustrated in FIG. 4, portion 302 c is blurred when compared toportions 302 a and 302 b. Here, the blurring of portion 302 c isillustrated by portion 302 c being shaded a lighter shade than it wasdepicted in FIG. 3. In contrast, portions 302 a and 302 b contain littleto no blur, which is illustrated by portions 302 a-302 b being shadedthe same color as they were illustrated in FIG. 3. Portion 302 c hasmore blur than portions 302 a and 302 b because portion 302 c is beingdisplayed via secondary display 206, which can have a lower maximumresolution than the maximum resolution of primary display 204 that isdisplaying portions 302 a-302 b.

When device 202 is worn by a user, the edge between the user's centralvision and peripheral vision typically lies close to area 404. Asdepicted in FIG. 4, area 404 corresponds to the areas of primary display204 and secondary display 206 that displays sections of portions 302 band 302 c.

Because the portion of 302 c is blurred in comparison to portion 302 b,hard clipping can occur in area 404 and/or the edge between the user'scentral and peripheral visions. In some embodiments, hard clippingoccurs when there is visual distortion between a portion of an objectdisplayed on one display and a portion of an object displayed on anotherdisplay. As further described below, hard clipping can occur when anobject transitions from one display to another display.

In some embodiments, the hard clipping in area 404 is an unintendedvisual effect. In particular, hard clipping can cause the user interfacedisplayed by device 202 to be distracting to a user wearing device 202.For example, the user interface can be distracting to a user wearingdevice 202 when objects are entering or leaving the user's centralvision.

FIG. 5 depicts exemplary regions on a device that has additive displaysin accordance with some embodiments. As further explained below, hardclipping may occur between one or more exemplary regions on the device.Because the field-of-view of primary display 204 is typically in auser's central vision and the field-of-view of secondary display 206 isin the user's peripheral vision, reducing hard clipping of objectstransitioning between primary display 204 and secondary display 206reduces high-frequency noise (e.g., vision snow, random and/or abruptfluctuations of objects, irregular movement or spatial relationshipsbetween objects) in the user's central and/or peripheral vision.Reducing high-frequency noise in the user's central and/or peripheralvision is advantageous because the high-frequency noise can cause a userto look unnecessarily at objects in the user's peripheral vision.Therefore, high-frequency noise can impact a user's ability to focus onobjects within the user's central vision, which can impact the abilityof device 202 to track and/or make accurate determinations based on theeye movement of the user.

FIG. 5 illustrates device 202 having exemplary first region 504 a andexemplary second region 504 b of primary display 204. First region 504 aextends across an area of primary display 204 from edge 208 of primarydisplay 204 to distance 510 a from edge 208 of primary display 204. Insome embodiments, at least a portion of area 404 depicted in FIG. 4overlaps a portion of first region 504 a depicted in FIG. 5. In someembodiments, at least a portion of the edge of a user's central andperipheral visions lies in a portion of first region 504 a when the useris wearing device 202.

Second region 504 b extends across the area of primary display 204 fromthe center of primary display 204 to distance 510 b towards the edge 208of primary display 204. In some embodiments, no portion of area 404depicted in FIG. 4 overlaps a portion of first region 504 b depicted inFIG. 5. In some embodiments, no portion of the edge of a user's centraland peripheral visions lies in a portion of first region 504 a when theuser is wearing device 202.

In some embodiments, first region 504 a and/or the second region 504 bextend across other areas of primary display 204 that are notillustrated by FIG. 5 or do not extend across areas illustrated by FIG.5. In some embodiments, first region 504 a and/or second region 504 bextends across the entirety of primary display 204.

FIGS. 6A-6C and FIG. 7 depict an exemplary technique for displaying avirtual object in accordance with some embodiments. Notably, in FIGS.6A-6C and FIG. 7, the exemplary technique and processes described belowcan reduce hard clipping (e.g., an abrupt change when an objecttransitions from one display to another display) of an object when theobject is transitioned between displays with different resolutions(e.g., between primary display 204 and secondary display 206) as shown,for example, in FIG. 4 above. As mentioned above, reducing hard clippingof objects transitioning between primary display 204 and secondarydisplay 206 reduces high-frequency noise in the user's central and/orperipheral visions. Reducing high-frequency noise in the user's centraland/or peripheral vision is advantageous because high-frequency noisecan impact the ability of device 202 to track and/or make accuratedeterminations based on the eye movement of the user.

FIG. 6A illustrates an environment that includes device 202 and pencil302. At FIG. 6A, user 250 is wearing device 202 as described above inrelation to FIG. 2B. However, for ease of discussion, user 250 is notdepicted in FIG. 6A.

FIG. 6A depicts device 202 displaying pencil 302 falling from a positionthat is outside of the field of view of primary display 204 andsecondary display 206. In FIGS. 6A-6C, the portions of pencil 302 thatare represented by dashed lines correspond to the portions of 302 thatcannot be seen by a user looking through primary display 204 and/orsecondary display 206 while wearing device 202.

As illustrated in FIG. 6A, pencil 302 has fallen from the position thatis outside of the field-of-views of primary display 204 and secondarydisplay 206 to a position that is inside the field-of-view of secondarydisplay 206 and outside of the field-of-view of primary display 204. InFIG. 6A, portion 302 a is presented via secondary display 206. In someembodiments, device 202 generates a virtual representation of portion302 a and displays, via secondary display 206, the virtualrepresentation.

In FIG. 6A, device 202 displays the virtual representation of portion302 a that is presented at a resolution that corresponds to the maximumresolution of secondary display 206. As illustrated in FIG. 6A, portion302 a is blurred (e.g., illustrated by its lighter shade) when comparedto portion 302 a in FIG. 3. In some embodiments, the virtualrepresentation of portion 302 a displayed in FIG. 6A has no visualeffect applied to create change in blur.

Some period of time following what is depicted in FIG. 6A, as pencil 302continues to fall, FIG. 6B depicts portion 302 a having fallen within aparticular region of primary display 204 (e.g., region 504 a asdescribed below in FIG. 5). While displaying pencil 302 via secondarydisplay 206 (e.g., inside the field-of-view of secondary display 206),pencil 302 is determined to have fallen to a location that is withindistance 510 a from edge 208 of primary display 204 (e.g., moving intoprimary display 204), as illustrated in FIG. 6B. Based on thisdetermination, device 202 generates a virtual representation by applyinga visual effect to portion 302 a of pencil 302 and displays the virtualrepresentation in primary display 204, as illustrated in FIG. 6B.

Here, the virtual representation that corresponds to portion 302 a isblurred by applying the visual effect to portion 302 a. As illustratedin FIG. 6B, applying the visual effect causes portion 302 a to bedisplayed as a gradient (e.g., from a light color as shown to a darkcolor) to provide a smoother transition at the edge of the user'scentral and peripheral visions (and/or area 404 discussed in FIG. 4). Insome embodiments, after applying the visual effect to portion 302 a, thevirtual representation that corresponds to portion 302 a is faded. Insome embodiments, the virtual representation is faded inverselyproportional to a detected amount of blur of secondary display 206 (orblur of at least one region of secondary display 206) after applying thevisual effect to portion 302 a. In some embodiments, the virtualrepresentation does not include a gradient after applying the visualeffect to portion 302 a. In other embodiments, applying the visualeffect includes applying a blurring function to portion 302 a. In someembodiments, the amount of blur applied to portion 302 a at edge 208 canbe selected to match the blur of secondary display 206. In doing so, theappearance of virtual object 302 on opposite sides of edge 208 may bethe same or similar, thereby reducing the hard clipping effect discussedabove. The amount of blur applied to portion 302 a (or other virtualcontent) within portion 504 a can be reduced based on the distance fromedge 208, with no blur being applied at distance D1 from edge 208. Theblur can be reduced linearly or non-linearly as a function of thedistance from edge 208.

As illustrated in FIG. 6B, portion 302 b has also moved into thefield-of-view of secondary display 206. Thus, in addition to displayinga virtual representation that corresponds to portion 302 a in primarydisplay 204, device 202 also displays a virtual representation thatcorresponds to portion 302 b of pencil 302. In some embodiments, device202 displays the virtual representation that corresponds to portion 302b using the techniques as described above in relation to displaying thevirtual representation that corresponded to the portion 302 a in FIG.6A.

Notably, as illustrated in FIG. 6B, the virtual representations thatcorrespond to portion 302 a and 302 b are blurred, where the blurring isrepresented by the difference in shading of respective virtualrepresentations that correspond to the respective portions of pencil302. In FIG. 6B, the virtual representation that corresponds to portion302 b has more blur than the virtual representation that corresponds toportion 302 a.

In FIG. 6B, the virtual representation that corresponds to portion 302 bis blurred because it is displayed via secondary display 206, which is alower-resolution display with more blur than primary display 204. On theother hand, the virtual representation that corresponds to portion 302 ais blurred because device 202 applied a visual effect to portion 302 abefore displaying the virtual representation that corresponds to portion302 a via primary display 204. Thus, the virtual representation thatcorresponds to portion 302 a with the visual effect applied is moresimilar to (e.g., has a closer blur value to) the virtual representationthat corresponds to portion 302 b than a virtual representation thatcorresponds to portion 302 a that has no visual effect applied.

In some embodiments, device 202 applies a visual effect to portion 302 asuch that the virtual representation of portion 302 a appears tovisually match the virtual representation that corresponds to portion302 b displayed in secondary display 206 (e.g., where no visual effecthas been applied to portion 302 b). Because secondary display 206 has alower resolution than primary display 204, device 202 applies the visualeffect to portion 302 a to make the visual appearance of the virtualrepresentation that corresponds to portion 302 a match the visualappearance of the virtual representation that corresponds to portion 302b.

In some embodiments, device 202 applies the visual effect to portion 302to smooth out the transition of displaying the virtual representationthat corresponds to portion 302 a in FIG. 6A via a display having alower resolution (e.g., secondary display) to displaying the virtualrepresentation that corresponds to portion 302 a in FIG. 6B via adisplay having a higher resolution (e.g., primary display 204).

As illustrated in FIG. 6C, pencil 302 has fallen to a new position thatis inside the field-of-views of primary display 204 and secondarydisplay 206. As illustrated in FIG. 6C, portion 302 a is a distance(e.g., a distance that is the sum of distance 510 a and distance 510 b)from edge 208 of primary display 204, which is outside of distance 510a. Because portion 302 a is outside of distance 510 a from edge 208 ofprimary display 204, portion 302 a has entered a second region (e.g.,region 504 b in FIG. 5) of primary display 204 that is outside of thefirst region of primary display 204. And, thus, no visual effect isapplied to the virtual representation of portion 302 a, which lies insecond region 504 b (shown in FIG. 5). In some embodiments, no visualeffect (or a different visual effect than the visual effect applied tothe first region) is applied to representations of portions of virtualobjects in the second region 504 b of primary display 204. In someembodiments, no visual effect is applied because portion 302 a is withinthe user's central vision, and device 202 has determined that the useris likely to be distracted by a virtual object (e.g., or a particularvisual object) that does not have a visual effect applied. Asillustrated in FIG. 6C, device 202 displays a virtual representation ofportion 302 c, a new portion of pencil 302 that is in the field-of-viewof secondary display 206, using the techniques as described above inrelation to display of the virtual representation of portion 302 a inFIG. 6A. Further, at FIG. 6C, device 202 displays the virtualrepresentation of portion 302 b with the visual effect applied (e.g.,using techniques similar to those described above in relation to portion302 a in FIG. 6C) because portion 302 b is within distance 510 a fromedge 208 of primary display 204.

In some embodiments, when the entirety of an object (e.g., pencil 302)is in the field-of-view of the primary display (e.g., no portion of theobject is in the field-of-view of the secondary display), no visualeffect is applied to any portion of a representation of the virtualobject, including a portion of a representation of the virtual objectdisplayed in second region 504 b described above with respect to FIG. 5.

In some embodiments, device 202 can determine to display differentvirtual objects differently. In some embodiments, device 202 can apply avisual effect to one virtual object while not applying a visual effectto another virtual object. In some embodiments, device 202 can applydifferent visual effects to virtual objects. In some embodiments, device202 uses different determinations to apply a visual effect to differentsets of virtual objects. In some embodiments, device 202 can apply avisual effect to one type of virtual object based on a determinationthat the virtual object has entered a region of the display but forgoapplying the visual effect to another type of virtual object based on adetermination that the other type of virtual object has entered theregion.

In some embodiments, the operations described in FIGS. 6A-6C can bedescribed in reverse (e.g., FIGS. 6C-6A). For example, a virtual objectcan transition from being within the field-of-views of the primarydisplay 204 and/or secondary display 206 to being out of thefield-of-views of primary display 204 and secondary display 206, usingsimilar techniques to those described above.

FIG. 7 is a flow diagram illustrating method 700 displaying a virtualobject in accordance with some embodiments. In some embodiments, themethod is performed by system 100 (FIGS. 1A and 1B). In someembodiments, the method is performed by device 202 (FIG. 2A). In someembodiments, the method is performed by a third device or system that isdifferent from device 202 or system 100. In some embodiments, the methodis performed by a combination of one or more of system 100, device 202,and the third device or system.

At block 702, a first portion of a virtual object is displayed via thesecondary display. For example, as described above with reference toFIG. 6C, a first portion 302 c of virtual object 302 is displayed viasecondary display 206. In some embodiments, at a first instance in time,the virtual object is presented on the secondary display (e.g.,secondary display 206) without displaying the virtual object on theprimary display (e.g., primary display 204). In some embodiments, whenthe virtual object is presented on the secondary display and while thevirtual object is not presented on the primary display, the virtualobject is in a field-of-view of the secondary display while the virtualobject is not within the field-of-view of the primary display.

With reference to FIG. 7, at block 704, a second portion of the virtualobject is displayed via the primary display. For example, as describedabove with reference to FIG. 6C, a second portion 302 b of virtualobject 302 is displayed via primary display 204. In some embodiments,the second portion of the virtual object is displayed via the primarydisplay when the device has moved within a predefined distance from thevirtual object. In some embodiments, the second portion of the virtualobject is displayed via the primary display when the virtual objectmoves within a predefined distance from the edge of the primary displayor within a predefined distance from the field-of-view (e.g., the edgeof the field-of-view) of the primary display.

With reference to FIG. 7, as part of displaying a second portion of thevirtual object (block 704), at block 706, in accordance with adetermination that the second portion of the virtual object is within apredefined distance from an edge of the primary display, a visual effectis applied to the second portion of the virtual object. For example, asdescribed above with reference to FIG. 6C, as part of displaying secondportion 302 b of virtual object 302, in accordance with a determinationthat second portion 302 b is within a predefined distance 510 a from anedge of primary display 204, a visual effect is applied to secondportion 302 b of virtual object 302.

In some embodiments, the visual effect is applied to portions of one ormore virtual objects while the portions of the one or more virtualobjects are in a first region of the primary display. For example, asdescribed above in relation to FIG. 6C, second portion 302 b has avisual effect applied while displayed in region 504 a that isillustrated in FIG. 5. In some embodiments, region 504 a is within apredefined distance from an edge of the primary display that is adjacentto an edge of the secondary display.

In some embodiments, the amount of visual effect applied can be the samefor two or more virtual objects in a particular region. Alternatively,in some embodiments, the amount of visual effect applied to a particularvirtual object is dynamic. For example, the amount of visual effectapplied can be based on one or more detected characteristics associatedwith the user wearing the device or based on one or more user settings.In some embodiments, user characteristics may include the user's pupilsize, pupil direction, pupil location, eye dominance, rate of blinking,etc. at a different point in time.

In some embodiments, the amount of visual effect applied is dependent onone or more characteristics of the virtual object. For example, in someembodiments, the amount of visual effect applied depends on the size,movement (e.g., velocity, acceleration), color, etc. of the virtualobject. In some embodiments, the amount of visual effect applied issmaller for a virtual object that is moving at a lower velocity andgreater for a virtual object that is moving at a higher velocity.

In some embodiments, the amount of visual effect applied is dependent onone or more characteristics of the device, such as the velocity oracceleration of the device. In some embodiments, the amount of visualeffect applied is smaller when the device is moving at a lower velocityand greater for when the device is moving at a greater velocity.

In some embodiments, while the visual effect is applied to a portion ofa virtual object when it is displayed in one region of the primarydisplay, the visual effect is not applied to another portion of thevirtual object that is displayed in another region of the primarydisplay. For example, as described above in relation to FIG. 6C, portion302 a does not have a visual effect applied while displayed in region504 b that is illustrated in FIG. 5.

In some embodiments, applying the visual effect to the second portion ofthe virtual object includes blurring the second portion of the virtualobject. In some embodiments, before applying the visual effect to thesecond portion of the virtual object, the second portion of the virtualobject already has a visual appearance that includes a first blurredappearance. In some embodiments, after applying the visual effect to thesecond portion of the virtual object, the second portion of the virtualobject has a subsequent visual appearance that includes a second blurredappearance that is different from (e.g., blurrier than) the firstblurred appearance.

In some embodiments, the second portion of the virtual object is blurredbased on a characteristic of a detected, via the one or more imagessensors, pupil aligned with a portion of primary and/or secondarydisplay. In some embodiments, a portion of the virtual object is blurredor faded according to one or more characteristics of the detected pupilwhile another portion of the virtual object is not. In some embodiments,the blurring or fading of a portion of the virtual object increases whenthe detected pupil is farther away from the edge of the primary display.

In some embodiments, after the blurring the second portion of thevirtual object, the second portion of the virtual object includes anamount of blur that is based on an amount of blur of the first portionof the virtual object. In some embodiments, the amount of blur (e.g., ofthe portion of the virtual object) matches the blur of the first portionof the virtual object. In some embodiments, as a part of blurring thesecond portion of the virtual object, an amount of blur of the secondportion of the virtual object is changed (e.g., continuously as thedistance between the object and device 202 changes) based on arespective distance between the virtual object and the secondarydisplay.

In some embodiments, as a part of applying the visual effect to thesecond portion of the virtual object, the second portion of the virtualobject is faded (e.g., faded in or out). In some embodiments, fading thesecond portion of the virtual object includes feathering an edge of theprimary display and/or the virtual object that is adjacent to the secondregion.

In some embodiments, before applying the visual effect to the secondportion of the virtual object, the second portion of the virtual objectalready has a visual appearance that includes a first faded appearance.In some embodiments, after applying the visual effect to the secondportion of the virtual object, the second portion of the virtual objecthas a subsequent visual appearance that includes a second fadedappearance that is different from (e.g., more faded than) the firstfaded appearance.

In some embodiments, the second portion of the virtual object is fadedproportionally to a blur of the secondary display (e.g., display 206)(e.g., at least one region of the secondary display). In someembodiments, the second portion of the virtual object is faded out to alevel that is inversely proportional to the blur of the secondarydisplay.

In some embodiments, the second portion of the virtual object is fadedfrom a first color to a second color. In some embodiments, the secondportion of the virtual object is faded, such that a gradient is appliedto the second portion of the virtual object. In some embodiments, whenthe gradient is applied to the second portion of the virtual object, thesecond portion of the virtual object includes a gradient (e.g., agradient that transitions from a first color to a second color).

In some embodiments, as a part of applying the visual effect to thesecond portion of the virtual object, a corner of the second portion ofthe virtual object is rounded. In some embodiments, the edges betweenthe primary display and the secondary display (and/or the first region)are rounded or modified to be displayed as rounded. In some embodiments,the corner is faded or has a particular visual effect applied (e.g., avisual effect is applied to the corner that fades the corner while otherportions of the second portion of the virtual object are not faded or donot have the particular visual effect applied). In some embodiments, agradient is applied to the corner when the corner is faded.

With reference again to FIG. 7, as part of displaying the second portion(block 704), in accordance with a determination that the second portionof the virtual object is not within a predefined distance from an edgeof the primary display, applying a visual effect to the second portionof the virtual object is forgone. For example, as described above withreference to FIG. 6C displaying second portion 302 c of virtual object302 via primary display 204, in accordance with a determination thatsecond portion 302 c of virtual object 302 is not within predefineddistance 510 a from an edge of primary display 204, applying a visualeffect to second portion 302 c of virtual object 302 is forgone.

FIGS. 8A-8B, 9A-9B, 10A-10B, 11A-11B, 12A-12B, 13A-13B, and 14A-14Bdepict exemplary techniques for transitioning an object between displaysin accordance with some embodiments. Among other benefits, thetechniques and processes described below help to conserve the batterylife of device 202 while continuing to maximize the ability of device202 to present information (e.g., present information to user 250 inFIG. 2B). Device 202 conserves battery life by intelligently shiftingthe presentation of information from primary display 204 to one or moreof secondary display 206 and a tertiary display (e.g., the LEDs aroundframe 210). Shifting the information preserves battery life becausesecondary display 206 and the tertiary display can have a maximumresolution that is lower than the resolution of primary display 204.Therefore, secondary display 206 and the tertiary display could use lessenergy than primary display 204 to display information.

As discussed below, FIGS. 8A, 9A, 10A, 11A, 12A, 13A, and 14A depict anexemplary scenario in which a user 250 interacts with an object 806while wearing device 202. However, this particular scenario is providedfor exemplary purposes only. The techniques discussed below or similartechniques can also be effectively applied to various other scenarios.

As mentioned above, FIG. 8A illustrates a scenario where user 250 iswearing device 202. In this scenario, device 202 is positioned distance804 a away from object 806, such as a set of keys. In some embodiments,object 806 is a physical article or physical object that is located in aphysical environment with user 250. In some embodiments, object 806 is avirtual object that is located in a CGR environment that is beingdisplayed to user 250 through device 202.

In some embodiments, object 806 moves while user 250 and/or device 202are stationary or, alternatively, non-stationary.

In some embodiments, the environment includes one or more objects otherthan object 806. In some embodiments, device 202 detects that object 806is in the environment shown in FIG. 8A. In some embodiments, device 202determines that the object is useful or an important object (e.g., ormore useful or more important than other objects in the environment). Insome embodiments, device 202 can use one or more algorithms and/or oneor more user preferences to determine whether an object is useful orimportant to a user or an application that corresponds to a particularobject (e.g., an application that is in communication with device 202).In some embodiments, device 202 receives a request or a notificationconcerning an object and bases this determination on whether the objectis useful or important based on the request or notification.

FIG. 8B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 8A. Device 202 includes primary display 204and secondary display 206 (in addition to one or more components asdescribed in FIG. 2A above). At FIG. 8B, primary display 204 andsecondary display 206 are not displaying (or presenting) any visualinformation related to directional information that corresponds to thelocation of object 806 relative to device 202 (directional information).In some embodiments, at least some visual information is not displayedvia primary display 204 and/or secondary display 206 because device 202is too far from object 806 (or distance 804 a is above a predeterminedthreshold distance).

As illustrated in FIG. 8B, device 202 further includes LEDs 810 a-810 fand 812 a-812 f (the LEDs). At FIG. 8B, the LEDs are not emitting light(e.g., emitting light to show directional information). In someembodiments, one or more of the LEDs are not emitting light becausedevice 202 is too far from object 806 (or distance 804 a is above apredetermined threshold distance).

FIG. 9A illustrates object 806 and/or device 202 having moved from theirpositions in FIG. 8A to their positions in FIG. 9A.

FIG. 9B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 9A. With reference to FIGS. 9A and 9B, assumethat when device 202 is distance 804 b away from object 806, device 202receives directional information. In response to receiving thedirectional information, device 202 determines that object 806 is a typeof object where directional information can be (e.g., is configured tobe, is determined to be) displayed via one or more of the LEDs. In someembodiments, device 202 makes this determination because device 202 isdistance 804 b away from object 806 (or distance 804 b is less than apredetermined threshold distance (e.g., a threshold distance thatcorresponds to a maximum distance to display directional information).

As illustrated in FIG. 9B, device 202 causes LEDs 812 a-812 f to emitlight in a pattern to show directional information because device 202 isdistance 804 b away from object 806 (or that object 806 is a type ofobject where directional information can be displayed via one or more ofthe LEDs). As illustrated in FIG. 9B, LEDs 812 a-812 f are on the rightside of device 202 in a direction that is more upward than downward aspositioned on frame 210 of device 202. Thus, LEDs 812 a-812 f indicatethat object 806 is in front of and to the right of device 202, whichmatches the general direction of the path between device 202 and object806 in FIG. 9A.

In FIG. 9B, LEDs 812 a and 812 f are hatched while LEDs 812 b-812 e aresolid to demonstrate a gradient field of light. Here, the gradient fieldof light can show stronger points (e.g., solid LEDs 812 b-812 e) of thedirectional pull of the directional information between object 806 anddevice 202 from the weaker points (e.g., hatched LEDs 812 a, 8120 of thedirectional pull. Notably, the gradient field can provide moredirectional information to a user, which can help the user navigate moreefficiently towards object 806.

In some embodiments, directional information is displayed via the LEDsrather than primary display 204 and/or secondary display 206 atdistances greater than or equal to a predetermined threshold distance(e.g., a distance that is greater than or equal to distance 804 b). Insome embodiments, directional information is displayed via the LEDsrather than primary display 204 and/or secondary display 206 becausedisplaying directional information via the LEDs instead of primarydisplay 204 and/or secondary display 206 reserves battery power.

In some embodiments, directional information is displayed via the LEDsrather than primary display 204 and/or secondary display 206 based onthe type of information that is permitted to be displayed when device202 is a certain distance away from object 806. In some embodiments,device 202 displays directional information that is not based on theappearance of the object (e.g., blinking LEDs) when device 202 isfurther away from object 806 (or within a first set of distances from anobject). In some embodiments, device 202 displays directionalinformation that is based on the appearance of the object (e.g., avisual representation of the object or a proxy object displayed viaprimary display 204 and/or secondary display 206) when device 202 iscloser to object 806 (or within a second set of distances away from anobject, where the second set of distances is different from the firstset of distances). In some embodiments, device 202 can displaydirectional information that is based on the appearance of the objectand directional information that is not based on the appearance of theobject (e.g., when device 202 is within a third set of distances awayfrom the object that is between the first and second set of distancesaway from the object. In some embodiments, directional information isdisplayed via the LEDs rather than the primary display 204 and/orsecondary display 206 based on whether or not the object is within theFOVs of the primary display 204 and/or secondary display 206. In someembodiments, directional information is displayed via the LEDs ratherthan the primary display 204 and/or secondary display 206 when theobject is a certain distance outside of the FOVs of primary display 204and/or secondary display 206.

FIG. 10A illustrates object 806 and/or device 202 having moved fromtheir positions in FIG. 9A to their positions in FIG. 10A. Inparticular, in FIG. 10A, device 202 is positioned distance 804 c fromobject 806, where distance 804 c is different than distance 804 b inFIG. 9A.

FIG. 10B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 10A. With reference to FIGS. 10A and 10B,assume device 202 receives directional information. In response toreceiving the directional information, using similar techniques to thosediscussed above with respect to FIGS. 9A and 9B, device 202 determinesthat object 806 is a type of object where directional information can bedisplayed via the LEDs (e.g., because device 202 is distance 804 c awayfrom object 806). Based on this determination, device 202 then causesLEDs 812 d-812 f to emit light that corresponds to the directionalinformation and LEDs 812 a-812 c to cease emitting light. Here, theupdated LEDs of device 202 show the newly received directionalinformation (e.g., showing directional information of the relative pathto the position of object 806 being down and to the left of the positionof device 202).

As illustrated in FIG. 10B, device 202 reduces the number of LEDsemitting light from the number of LEDs that were emitting light in FIG.9B. In some embodiments, when device 202 is closer to object 806 than itwas in FIG. 9B (e.g., in some embodiments, fewer LEDs can provide betterdirectional information when user 250 is closer to the object). In someembodiments, when there are physical and/or virtual structures orobjects between device 202 and object 806, device 202 updates theplurality of LEDs (and/or primary display 204 and/or secondary display206) of device 202 in succession (e.g., LEDs emitting light in apattern) such that user 250 can navigate around the physical and/orvirtual structures or objects between device 202 and object 806.

FIG. 11A illustrates object 806 and/or device 202 having moved fromtheir positions in FIG. 10A to their positions in FIG. 11A. Inparticular, in FIG. 11A, device 202 is positioned distance 804 d fromobject 806, where distance 804 d is different than distance 804 c inFIG. 10A.

FIG. 11B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 11A. With reference to FIGS. 11A and 11B,assume device 202 updates not only the LEDs but also secondary display206 because device 202 is closer (e.g., within a set of predeterminedthreshold distances) at distance 804 d away from object 806 (as opposedto device 202 being distance 804 c away from object 806 in FIG. 10B). Insome embodiments, device 202 updates not only the LEDs but alsosecondary display 206 because object 806 is within the FOV of thesecondary display 206.

Assume also that device 202 receives directional information. Inresponse to receiving the directional information, using similartechniques to those discussed above with respect to FIGS. 9A and 9B,device 202 determines that object 806 is a type of object wheredirectional information can be displayed via LED 812 e (e.g., becausedevice 202 is distance 804 d away from object 806). Based on thisdetermination, device 202 causes LED 812 e to emit light thatcorresponds to the directional information and LEDs 812 d and 812 f tocease emitting light, using one or more techniques described above inrelation to FIGS. 9B and 10B.

If device 202 determines that object 806 is a type of object wheredirectional information can be displayed via secondary display 206, thendevice 202 displays the directional information via secondary display206. Here, device 202 makes this determination because it is distance804 d away from object 806.

In some embodiments, because device 202 is distance 804 d away fromobject 806 and/or object 806 is within a predetermined thresholddistance outside the field-of-view of primary display 204, device 202causes secondary display 206 to display modified representation 826 a ofobject 806. Here, modified representation 826 a is a representation of aproxy object or a symbolic representation of object 806 and is visuallydistinguishable from object 806. As illustrated in FIG. 11B, modifiedrepresentation 826 a is a blob of a plurality of pixels at the edge ofsecondary display 206.

In some embodiments, object 806 is inside of the field-of-view ofsecondary display 206 when device 202 is distance 804 d away from object806. In some embodiments, object 806 is at a location inside of thefield-of-view of secondary display 206 that corresponds to the locationon secondary display 206 where modified representation 826 a isdisplayed.

In some embodiments, object 806 is outside of the field-of-view ofsecondary display 206 when device 202 is distance 804 d away from object806. Thus, in some embodiments, device 202 displays modifiedrepresentation 826 a, via secondary display 206, when object 806 isoutside of the field-of-view of secondary display 206. In someembodiments, displaying modified representation 826 a at a respectivelocation on secondary display 206 when object 806 is outside of thefield-of-view of secondary display 206 provides directional informationfor locating objects that are outside of the field-of-view of thedisplays of device 202.

In FIG. 11B, modified representation 826 a does not resemble an actualrepresentation of object 806. In some embodiments, modifiedrepresentation 826 a does not resemble object 806 because it does nothave two or more features, such as the size, shape, color, texture,blur, etc., of object 806.

Modified representation 826 a shows that an object (e.g., an object thatis useful or important as described above in relation to FIG. 8A) iswithin or outside of the user's peripheral vision (e.g., the user'speripheral vision receiving visual information presented or displayedvia secondary display 206). Modified representation 826 a also can showthe general direction of an object's position relative to the positionof device 202. In some embodiments, the general direction is shown bypositioning modified object representation 826 a at a particularlocation on secondary display 206.

In some embodiments, modified representation 826 a is static (e.g., notmoving, not pulsating, not animating) while the lights emitting from oneor more of the LEDs are not static (e.g., moving, pulsating, animating)or vice-versa. In some embodiments, modified representation 826 a andLEDs 812 a-812 l are both static (e.g., or not static).

FIG. 12A illustrates object 806 and/or device 202 having moved fromtheir positions in FIG. 11A to their positions in FIG. 12A. Inparticular, in FIG. 12A, device 202 is positioned distance 804 e fromobject 806, where distance 804 e is different than distance 804 d inFIG. 11A.

FIG. 12B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 12B. With reference to FIGS. 12A and 12B,assume device 202 receives new directional information. Here, based onthe new directional information, device 202 updates primary display 204and secondary display 206 to display representations of directionalinformation and forgoes displaying directional information via the LEDs.

In some embodiments, device 202 determines that object 806 is not a typeof object where directional information can be displayed via LEDs 812a-812 l because device 202 is distance 804 e away from object 806. Forexample, device 202 can determine that distance 804 e is a distancewhere device 202 is too close to object 806. In some embodiments, device202 ceases to display directional information the LEDs when directionalinformation or one or more types (e.g., representation 828) ofdirectional information are displayed via primary display 204 and/orsecondary display 206.

In some embodiments, in response to receiving the directionalinformation, device 202 determines that object 806 is a type of objectwhere directional information for the object can be displayed viasecondary display 512 (e.g., because device 202 is distance 804 e awayfrom object 806 and/or object 806 is within a predetermined thresholddistance outside the field-of-view of primary display 204), usingsimilar techniques as described above in FIG. 11B. Because of thisdetermination, device 202 causes secondary display 206 to displaymodified representation 826 b.

In FIG. 12B, modified representation 826 b is larger than modifiedrepresentation 826 a because device 202 is closer to object 806 in FIG.12B than it was in FIG. 11B. As illustrated in FIG. 12B, modifiedrepresentation 826 b is displayed in a new location on secondary display206 that is closer to primary display 204 than the location of modifiedrepresentation 826 a as shown in FIG. 11A. Modified representation 826 bis displayed in the new location because object 806 is closer to thefield-of-view of primary display 204.

In some embodiments, modified representation 826 b occupies a portion ofsecondary display 206 that modified representation 826 a occupied. Insome embodiments, modified representation 826 b grows from modifiedrepresentation 826 a (e.g., via an animation). In some embodiments,device 202 displays modified representation 826 b and/or modifiedrepresentation 826 a with a brightness that is based on the size ofobject 806. For example, device 202 displays a modified representationof the large object with a brightness that is lower or dimmer than thebrightness of a modified representation of a small object.

In some embodiments, in response to receiving the directionalinformation, device 202 determines that object 806 is a type of objectwhere directional information for the object can be displayed viaprimary display 204. At FIG. 12B, device 202 makes this determinationbased on a determination that distance 804 e is close enough to object806. Based on this determination, device 202 causes representation 828to be displayed via primary display 204.

In FIG. 12B, representation 828 is an arrow pointing to the position ofobject 806. In some embodiments, device 202 displays representation 828to indicate that user 250 can adjust the field-of-view of primarydisplay 204 (e.g., by moving the head of user 250 while standingstationary), such that object 806 can be located within thefield-of-view of primary display 204. In some embodiments, device 202causes representation 828 to be animated. In some embodiments, device202 causes representation 828 (or a representation that corresponds torepresentation 828) to be displayed via secondary display 206.

FIG. 13A illustrates object 806 and/or device 202 having moved fromtheir positions in FIG. 12A to their positions in FIG. 13A. Inparticular, in FIG. 13A, device 202 is positioned distance 804 f fromobject 806, where distance 804 f is different than distance 804 e inFIG. 12A. Moreover, as depicted in FIG. 13A, object 806 is within thefield of view of primary display 204 of device 202.

FIG. 13B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 13A. With reference to FIGS. 13A and 13B,assume device 202 receives new directional information. Because device202 is distance 804 f from object 806 and/or because at least a portionof object 806 is in the field-of-view of the primary display and in thefield-of-view of the secondary display, device 202 displays an actualrepresentation of a portion of object 806 via primary display 204 and amodified representation 826 c of object 806 via secondary display 206.In some embodiments, the actual representation of the portion of theobject resembles (e.g., has two or more features that are similar toobject 806), while the modified representation does not resemble object806. In some embodiments, the actual representation of the portion ofthe object is displayed using a first model of the object while themodified representation is displayed using a second model of the object.In some embodiments, the first model of the object is different from thesecond model of the object because the second model of the object issimpler than the first model of the object. In some embodiments, thesecond model of the object is simpler than the first model of the objectbecause the second model has less detail (e.g., a three-dimensionalmodel that has fewer polygons) than the first model.

In some embodiments, a portion of modified representation 826 b movesinto primary display 204 by applying a visual effect to a virtualrepresentation of a portion of object 806, using similar techniques tothose described above in FIG. 7 (e.g., to reduce the clipping of object806 transitioning between primary display 204 and secondary display206). In some embodiments, device 202 replaces modified representation826 b or 826 c with a representation of an actual portion of object 806on secondary display 206 when transitioning an actual representation ofa portion of object 806 to be displayed via primary display 204, usingsimilar techniques to those described above in FIG. 7.

FIG. 14A illustrates object 806 and/or device 202 having moved fromtheir positions in FIG. 13A to their positions in FIG. 14A. Inparticular, in FIG. 14A, device 202 is positioned distance 804 g fromobject 806, where distance 804 g is different than distance 804 f inFIG. 13A. Moreover, as depicted in FIG. 14A, the entirety of object 806is now in the field of view of primary display 204 of device 202.

FIG. 14B illustrates device 202 at a time that corresponds to thescenario depicted in FIG. 14A. With reference to FIGS. 14A and 14B,device 202 displays an actual representation of object 806 (or, in someembodiments, a portion of object 806) to be displayed via primarydisplay 204 and ceases to display a modified representation of object806 via secondary display 206. In some embodiments, device 202 causesthe actual representation of object 806 via primary display 204 andceases to display the modified representation of object 806 viasecondary display 206 because device 202 is within the field-of-view ofprimary display 204 and not within the field-of-view of secondarydisplay 206.

In some embodiments, at FIGS. 13B and 14B, device 202 can display aproxy representation of object 806 via secondary display 206 instead ofdisplaying the actual representation of object 806 via primary display204. In some embodiments, secondary display 206 can overlap primarydisplay 204, such that secondary display 206 covers most (or theentirety) of primary display 204. Thus, in some of these embodiments, aproxy representation of object 806 can be shown via the secondarydisplay 206 over a portion of primary display 204 until device 202receives a request to interact with object 806. For example, a requestto interact with object 806 can be received when an input (e.g., voiceinput, physical input) is determined to be directed to object 806 and/ora process is initiated to perform an operation that involves (e.g.,indirectly or directly) one or more components of object 806 and/or anobject associated with object 806. In some embodiments, in response toreceiving the request to interact with object 806, device 202 ceases todisplay the proxy representation of object 806 via secondary display 206and displays the actual representation of object 806 via primary display204. Thus, in embodiments where device 202 uses more battery power todisplay objects via primary display 204 than secondary display 206,device 202's battery power is saved because primary display 204 is notused to display a representation of object 806 until a determination hasbeen made that object 806 may be interacted with.

In some embodiments, at FIGS. 13B and 14B, device 202 can display aproxy representation of object 806 instead of displaying the actualrepresentation of object 806 via primary display 204. In someembodiments, the proxy representation of object 806 has less visualdetails and/or takes less power to display than the actualrepresentation of object 806. In some embodiments, while displaying theproxy representation of object 806 via primary display 204, device 202receives a request to interact with object 806. For example, a requestto interact with object 806 can be received when an input (e.g., voiceinput, physical input) is determined to be directed to object 806 and/ora process is initiated to perform an operation that involves (e.g.,indirectly or directly) one or more components of object 806 and/or anobject associated with object 806. In some embodiments, in response toreceiving the request to interact with object 806, device 202 replacesdisplay of the proxy representation of object 806 with the actualrepresentation of object 806. Thus, in some embodiments, battery poweris saved and/or the number of processes being performed by device 202 isreduced because the actual representation of object 806 is displayedonly after a determination has been made that object 806 may beinteracted with. In some embodiments, the LEDs are inactive (or notincluded as a part of device 202) while the proxy representation ofobject 806 and/or the actual representation of object 806 are displayed.

FIG. 15 is a flow diagram illustrating a method for transitioningobjects between displays based on directional information in accordancewith some embodiments. In some embodiments, the method is performed bysystem 100 (FIGS. 1A and 1B). In some embodiments, the method isperformed by device 202 (FIG. 2A). In some embodiments, the method isperformed by a third device or system that is different from device 202or system 100. In some embodiments, the method is performed by acombination of one or more of system 100, device 202, and the thirddevice or system.

At block 1502, directional information (e.g., 804 a-804 d) correspondingto a location (e.g., north, south, east, west, right, left, back,forward, other cardinal directions, or any combination thereof) of anobject that is outside a field-of-view of the primary display (e.g.,204) of the device and the secondary display of the device is received.In some embodiments, the object is a physical object or article in aphysical environment; in some embodiments, the object is a virtualobject in a virtual environment.

In some embodiments, the primary display, secondary display, and/ortertiary display are different types of displays. In some embodiments,the primary display is a wave guide display. In some embodiments, thesecondary display is an organic light-emitting diode display. In someembodiments, the tertiary display is a plurality of light-emittingdiodes.

At blocks 1504 and 1506, in response to receiving the directionalinformation corresponding to the location of the object that is outsidethe field-of-view of the primary display and the secondary display andin accordance with a determination that first criteria are satisfied, afirst representation of directional information is displayed via thesecondary display.

In some embodiments, the first criteria include a criterion that is metwhen the object is a type of object where directional information forthe object can be displayed in the secondary display. In someembodiments, directional information for the object is permitted to bedisplayed via the secondary display when the object is a predetermineddistance away from or beside a portion (e.g., primary display, secondarydisplay, the frame of the device). In some embodiments, directionalinformation for the object is permitted to be displayed via thesecondary display when the directional information is received from anapplication or the object corresponding to an application and the objectis configured by the application (or another process) to be displayedvia the secondary display, when the object has one or more properties orfeatures (e.g., size, shape, distance) that allow it to be displayed viathe secondary display). In some embodiments, directional information forthe object is permitted to be displayed via the secondary display, whenthe object is not in a particular location or direction, or the objectwould cause the representation of the object to be displayed adjacent toa portion of the device (e.g., the inside of the frame of the device,near the bridge of the device, near the nose of the user).

At blocks 1504 and 1508, in response to receiving the directionalinformation corresponding to the location of the object that is outsidethe field-of-view of the primary display and the secondary display andin accordance with a determination that second criteria are satisfied, asecond representation (e.g., 812 a-812 g) of directional information(e.g., 804 b-804 d) is displayed via the tertiary display.

In some embodiments, the second criteria include a criterion that is metwhen the object is a type of object where directional information forthe object can be displayed in the tertiary display. In someembodiments, directional information for the object is permitted to bedisplayed via the tertiary display when the object is a predetermineddistance away from or beside a portion (e.g., primary display, secondarydisplay, frame) of the device when the directional information isreceived from an application or the object corresponding to anapplication and the object is configured by the application (or anotherprocess) to be displayed via the tertiary display, when the object hasone or more properties or features (e.g., size, shape, distance) thatallow it to be displayed via the tertiary display, when the object isnot in a particular location or direction or when the object would causethe representation of the object to be displayed adjacent to a portionof the device (e.g., the inside of the frame of the device, near thebridge of the device, near the nose of the user) or in a portion of thetertiary display.

In some embodiments, the first representation is a virtual object (e.g.,first representation is a modified representation of the object), andthe second representation is not a virtual object within the CGRenvironment. In some embodiments, the first modified representation ofthe virtual object can be visually distinguished from an actualrepresentation of the object. For example, the first modifiedrepresentation can have a visual appearance that is not similar or doesnot look like the actual object, while the actual representation of theobject can have a visual appearance that is similar or does look likethe actual object. In some embodiments, a device will not be able toidentify, using image recognition techniques, the actual object whenprocessing the modified representation of the object, but will be ableto identify the actual object when processing the actual representationof the object. In some embodiments, the first modified representationcan be a representation that does not resemble (e.g., does not have twoor more of the same features such as color, shape, texture, size,brightness, boldness, fading) the actual object while the actualrepresentation is a representation that does resemble the actual object.

In some embodiments, the first representation is a staticrepresentation. For example, a static representation is a representationthat does not move or is not animated over a period of time (e.g., notchanging, not pulsating, not being in a predetermined sequence orpattern, etc.). In some embodiments, the second representation is adynamic representation. For example, a dynamic representation is arepresentation that does move or is animated over a period of time(e.g., pulsating, displayed in a predetermined sequence or pattern,changing colors over a period of time). In some embodiments, the firstrepresentation is a different color than the second representation. Insome embodiments, the first representation is a different size than thesecond representation. In some embodiments, the first representation isnot displayed while the second representation is displayed orvice-versa. In some embodiments, at least one of the firstrepresentation and the second representation includes a gradient (e.g.,a gradient field (e.g., solid and hatched LEDs) that shows direction).

In some embodiments, the primary display and the secondary display candisplay different representations of the object. In some embodiments,the first representation of directional information is a firstrepresentation of the object that is displayed via the secondarydisplay. In some embodiments, while the first representation ofdirectional information is displayed via the secondary display, theobject is detected inside of the field-of-view of the primary display ofthe HMD device. In some embodiments, in response to detecting that theobject is inside of the field-of-view of the primary display, a secondrepresentation of the object is displayed via the primary display. Insome embodiments, the first representation of the object, that isdisplayed via the secondary display, is displayed based on a first modelof the object. In some embodiments, the second representation of theobject, that is displayed via the primary display, is displayed based ona second model of the object. In some embodiments, the second model ofthe object is simpler (e.g., has more visual characteristics, morevisual details) than the first model of the object.

In some embodiments, the tertiary display circumscribes a first portionof the secondary display. In some embodiments, the tertiary displaycircumscribes the first portion of the secondary display while notcircumscribing a second portion of the secondary display. In someembodiments, the primary display overlaps the secondary display.

In some embodiments, a representation of the object that is differentfrom the first and second representations of directional information isdisplayed via the primary display while the first representation isdisplayed via the secondary display. In some embodiments, therepresentations of directional information are associated with thelocation of the object. For example, the representations of directionalinformation can indicate the position of a device relative to thelocation of the object.

In some embodiments, a representation of the object may not be displayedvia the primary display until the object is interacted with. In someembodiments, directional information corresponding to a location of anobject that is inside a field-of-view of the primary display of the HMDdevice and the secondary display of the HMD device is received. In someembodiments, in response to receiving the directional informationcorresponding to the location of the object that is inside thefield-of-view of the primary display of the HMD device and the secondarydisplay of the HMD device, a fourth representation of the object isdisplayed via the secondary display without a fifth representation ofthe object being displayed via the primary display. In some embodiments,a request to interact with (e.g., select the object, initiate a processthat involves the object) the object is received. In some embodiments,in response to receiving the request to interact with the object, thefifth representation of the object is displayed via the primary display,and the fourth representation of the object ceasing to be displayed viathe secondary display.

In some embodiments, a request to interact with the object is received.In some embodiments, a representation of the object is displayed via theprimary display in response to receiving the request to interact withthe object.

In some embodiments, while the first representation and the secondrepresentation are displayed, the first representation is displayed, viathe secondary display, in a first location that is in a respectivedirection relative to the primary display and the second representationis not displayed, via the tertiary display, in a second location that isin the respective direction relative to the primary display.

FIG. 16 is a flow diagram illustrating a method for displaying amodified representation of an object in accordance with someembodiments. In some embodiments, the method is performed by system 100(FIGS. 1A and 1B). In some embodiments, the method is performed bydevice 202 (FIG. 2A). In some embodiments, the method is performed by athird device or system that is different from device 202 or system 100.In some embodiments, the method is performed by a combination of one ormore of system 100, device 202, and the third device or system.

At block 1602, an object (e.g., a virtual object, a virtual object thatis a representation of a physical object, a physical object in aphysical environment) in a computer-generated reality (CGR) environmentat a first location (or a position) is detected. In some embodiments,the object is determined to be important or useful. For example, anobject can be determined to be important or useful by receiving userinput related to the object, by receiving user interaction that isrelated to the object, by receiving data identifying the object asimportant or useful, etc. In some embodiments, data identifying that theobject is important or useful is received from one or more applicationsin communication with or being executed by device 202.

At blocks 1604 and 1606, a first modified representation of the virtualobject is displayed in response to detecting the object in the CGRenvironment at the first location and in accordance with a determinationthat the first location is within a first predetermined distance outsideof the field-of-view of the primary display of the device.

The first modified representation of the virtual object can be visuallydistinguished from an actual representation of the object. For example,the first modified representation can have a visual appearance that isnot similar or does not look like the actual object, while the actualrepresentation of the object can have a visual appearance that issimilar or does look like the actual object. In some embodiments, adevice may not be able to identify, using image recognition techniques,the actual object when processing the modified representation of theobject, but will be able to identify the actual object when processingthe actual representation of the object. In some embodiments, the firstmodified representation can be a representation that does not resemble(e.g., does not have two or more of the same features, such as color,shape, texture, size, brightness, boldness, fading) the actual objectwhile the actual representation is a representation that does resemblethe actual object.

In some embodiments, the first modified representation has visualcontent (e.g., data that represents a displayed representation) that isdifferent from visual content of the first actual representation. Insome embodiments, the modified representation of the object has astructure that is different from the structure of the actual object, andthe actual representation of the object has the same structure as theobject. For example, if the actual object is a tree, the actualrepresentation of the object can visually look like a tree, while thefirst modified representation looks like a blob of pixels that is not inthe shape of a tree.

In some embodiments, the first modified representation has a size thatis based on the distance between the first location and thefield-of-view of the primary display of the device. In some embodiments,the first modified representation is a first size when the location ofthe object is a first distance outside of the primary display. In someembodiments, the first modified representation is a second size when thelocation of the object is a second distance outside of the primarydisplay. In some embodiments, the first modified representationdisplayed at the first size is bigger than the first modifiedrepresentation displayed at the second size when the first distance iscloser to the primary display than the second distance. In someembodiments, the size of the first modified representation increases insize when the object is detected at a location that is closer to thefield-of-view of the primary display of the device than the firstlocation.

In some embodiments, the first modified representation has a brightness(e.g., larger objects are made to be dimmer) that is based on the sizeof the object. In some embodiments, the first modified representationhas a first brightness when the object is a first size. In someembodiments, the first modified representation has a second brightnesswhen the object is a second size. In some embodiments, the firstmodified representation displayed at the first brightness is brighterthan the first modified representation displayed at the secondbrightness when the first size of the object is smaller than the secondsize of the object.

In some embodiments, the first modified representation has a brightnessthat is based on the size of the object relative to another object thatis displayed via a device. For example, the brightness of the firstmodified representation can increase as the size of the object getsgreater than the size of the other object, and vice-versa. In someembodiments, when the size of the object is substantially equal to thesize of the other object, the brightness of the modified representationof the object can be equal to the brightness of a representation of theother object. Thus, in some embodiments, as a user moves throughout anenvironment, the brightness of the modified representation of the objectcan change based on other objects within the environment.

In some embodiments, the first modified representation has a blur thatis higher than a blur of the first actual representation. In someembodiments, the first modified representation is faded more than thefirst actual representation.

In some embodiments, as a part of displaying the first modifiedrepresentation, an animation can be displayed. For example, theanimation can be a pulsating animation, an animation that shifts ormoves the first modified representation, an animation that transformsthe first actual representation of the object into the first modifiedrepresentation, an animation that fades in or fades out the firstmodified representation.

In some embodiments, a second actual representation of a first portionof the object is displayed via the primary display in response todetecting the object in the CGR environment at the first location and inaccordance with a determination that the first location is within thefield-of-view of the primary display and a field-of-view of thesecondary display. For example, as illustrated in FIG. 13B, the secondactual representation (e.g., 806) of the first portion of the object canbe displayed via the primary display when a device is within a thresholddistance inside of the field-of-view of the secondary display.

While displaying, via the primary display, the second actualrepresentation of the first portion of the object, a second modifiedrepresentation of a second portion of the object is displayed via thesecondary display. For example, as illustrated in FIG. 13B, the secondmodified representation (e.g., 826 c) is displayed via the secondarydisplay.

In some embodiments, the second portion of the object is a differentportion of the object than the first portion of the object. In someembodiments, the first portion of the object is within the field-of-viewof the primary display but not in the field-of-view of the secondarydisplay. In some embodiments, the second portion of the object is withinthe field-of-view of the secondary display but not in the field-of-viewof the primary display. In some embodiments, the second actualrepresentation is displayed at a higher resolution than the secondmodified representation. In some embodiments, the second actualrepresentation is displayed adjacent to an edge of the primary displaythat is adjacent to the secondary display. In some embodiments, thesecond modified representation is displayed adjacent to an edge of thesecondary display that is adjacent to the primary display.

In some embodiments, as a part of displaying, via the primary display,the second actual representation of the object, a visual effect isapplied to the second actual representation of the object. In someembodiments, the second actual representation of the object after thevisual effect is applied has a blur that is greater than a blur of thesecond actual representation of the object before the visual effect wasapplied. In some embodiments, one or more of the second actualrepresentation of the first portion of the object, displayed via theprimary display, and the second modified representation of the secondportion of the object, displayed via the secondary display, aredisplayed using one or more techniques discussed above in relation toFIG. 7.

In some embodiments, a representation that corresponds to directionalinformation to locate the object is displayed via the primary display inresponse to detecting the object in the CGR environment at the firstlocation and in accordance with a determination that the first locationis within a second predetermined distance outside of the field-of-viewof the primary display of the device. For example, as illustrated inFIGS. 12A and 12B, the representation (e.g., 828) that corresponds todirectional information to locate the object is displayed via theprimary display when the distance between user 250 and object 806 isdistance 804 e away from each other, which is different from user 250and object 806 being distance 804 d away from each other in FIG. 11A.

In some embodiments, the representation that corresponds to directionalinformation to locate the object is different from the actualrepresentation. The representation that corresponds to the directionalinformation to locate the object may provide an indication of thelocation of the object, the closeness of the object to the device,whether the device is moving away from or closer to the object.

In some embodiments, while displaying, via the primary display, therepresentation that corresponds to the location of the object, amodified representation of the object is displayed via the secondarydisplay.

In some embodiments, in response to detecting the object at the firstlocation and in accordance with a determination that the first locationis within the field-of-view of the primary display and not within thefield-of-view of the secondary display, the first actual representation(e.g., 806) is displayed via the primary display, while the firstmodified representation ceases to be displayed via the secondarydisplay. In some embodiments, a portion of the first actualrepresentation of the object is displayed. In some embodiments, theentirety of the object is displayed.

FIGS. 17A-17B, 18, and 25 depict exemplary techniques for managing oneor more displays based on data associated with one or more processes inaccordance with some embodiments. In FIG. 17A, device 202 is depictedwith one or more components, as discussed above in FIGS. 2A-2B and FIG.8A.

FIG. 17A depicts device 202 displaying a plurality of application icons.The plurality of application icons include messaging application icon1702 a, health application icon 1702 b, phone application icon 1702 c,and e-mail application icon 1702 d.

With reference to FIG. 17A, assume that device 202 receives a newmessage via a messaging application that corresponds to messaging icon1702 a. In response to receiving the new message, device 202 candetermine if the information related to the new message will bedisplayed via a particular display (e.g., primary display 204, secondarydisplay 206, the LEDs, etc.).

In some embodiments, in response to receiving the new message, device202 determines that information related to the new message is permitted(or configured) to be displayed via secondary display 206. As depictedin FIG. 17B, device 202 updates messaging application icon 1702 a toinclude message status indicator 1702 a 1 because of this determination.As shown in FIG. 17B, message status indicator 1702 a 1 is a “1”adjacent to messaging icon 1702 a and indicates that one additional newor unread message has been received. An unread message can include amessage that has not received any user interaction, such as a voicemailmessage that has not been heard, a text or e-mail message that has notbeen read, a notification or alert from an application that has not beenopened, a notification or alert that has not been deleted, etc.

In some embodiments, in response to receiving the new message, device202 determines that information related to the new message is permitted(or configured) to be displayed via a tertiary display, such as theLEDs. As depicted in FIG. 17B, device 202 causes the LEDs to emit lightbecause of this determination. In some embodiments, device 202 causesonly a portion of the LEDs to emit light.

One or more other types of alerts, notifications, sensor data, requestsor data from an application, incoming phone calls, etc. can be received.In some embodiments, device 202 performs similar actions in response totheir receipt, as described above in relation to the new messagereceived in FIGS. 17A-17B. For example, device 202 can cause the LEDs toemit light when it is determined that health data detected by a heartrate sensor (or one or more other sensors that detect personal or healthdata) is above or below a threshold level and that information relatingthe data is permitted to be displayed via the LEDs.

Device 202 can also perform other actions in response to receiving oneor more messages, alerts, notifications, sensor data, requests or datafrom an application, incoming phone calls, etc. For example, as depictedin FIG. 18, device 202 displays a color across at least a portion ofsecondary display 206 based on the determination that informationrelated to the new message is permitted to be displayed via secondarydisplay 206. In some embodiments, a portion of primary display 204 isupdated because of this determination.

In some embodiments, device 202 can display an animation via one or moredisplays. In some embodiments, the animation can include one or morepulsating colors, one or more moving objects, one or more objectschanging shape or transitioning into one or more other objects, etc. Forexample, device 202 can display an animation of a pulsating color onprimary display 204 and/or secondary display 206 based on adetermination that health data of a user is above/below a thresholdlevel and that information relating to the health data is permitted tobe displayed on the respective display.

FIGS. 19A-19B and 25 depict an exemplary technique for managing one ormore displays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 19A depicts device 202 displaying a CGR environment that includesrepresentations of a plurality of objects 1910 a-1910 d. Objects 1910a-1910 d are cars, which are in the field-of-views of primary display204 and secondary display 206. Here, objects 1910 a-1910 d arerepresentative of physical cars in a physical environment that areviewed through the primary 204 and secondary 206 displays of device 202.In some embodiments, objects 1910 a-1910 d are representative of virtualobjects that are displayed by primary 204 and secondary 206 displays ofdevice 202.

With reference to FIG. 19A, assume device 202 receives data from anapplication that indicates that object 1910 a is associated with theuser of device 202. For example, in FIG. 19A, the application can be acar-sharing application, taxi application, car location application,etc. In some embodiments, device 202 determines that object 1910 a isassociated with the user of device 202 without receiving data from adedicated application using one or more user settings, data stored inmemory, one or more machine learning algorithms, etc.

In response to receiving the data from the application, device 202 candetermine if the information related to the data from the applicationwill be displayed via a particular display. In some embodiments, inresponse to receiving the data from the application, device 202determines that information related to the data is permitted to bedisplayed via secondary display 206. As depicted in FIG. 19B, based onthis determination, device 202 displays indicator 1912 around object1910 a on secondary display 206.

Indicator 1912 is displayed around object 1910 a to visually distinguishobject 1910 a from objects 1910 b-1910 d. In some embodiments, device202 causes object 1910 a to be visually distinguished from the otherobjects in other ways. For example, device 202 can cease to displayobjects 1910 b-1910 d, minimize objects 1910 b-1910 d, enlarge object1910 a, highlight object 1910 a, etc.

In some embodiments, in response to receiving the data from theapplication, device 202 determines that information related to the datais permitted to be displayed via the LEDs. As depicted in FIG. 19B,device 202 causes one or more of the LEDs to emit light based on thisdetermination.

FIGS. 20A-20D and 25 depict an exemplary technique for managing one ormore displays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 20A depicts device 202 displaying a CGR environment that includesrepresentation of time 2002. Representation of time 2002 is displayedvia primary display 204 and indicates that a process has a remainingtime of 60 seconds. In some embodiments, representation of time 2002 isrepresentative of a process associated with a timer application,delivery application, workout application, meeting application,productivity application, etc. In some embodiments, representation oftime 2002 is displayed via primary display 204 because a determinationhas been made that representation of time 2002 is permitted to bedisplayed via primary display 204.

Concurrently with representation of time 2002, representation of thetime 2004 is displayed via secondary display 2006. Representation oftime 2004 includes a colored overlay that covers the entirety ofsecondary display 206 to indicate a time remaining of 60 seconds. Insome embodiments, representation of time 2004 indicates the timerepresented by representation of time 2002. In other words, in someembodiments, a representation displayed via one display can correspondto or be representative of a representation displayed via anotherdisplay.

In some embodiments, representation of time 2002 is displayed viasecondary display 206 because a determination has been made thatrepresentation of time 2004 is permitted to be displayed via secondarydisplay 206.

As depicted in FIG. 20A, representation of time 2004 indicates apercentage of time remaining when compared to an initiation time of atimer or process. For example, representation 2004 indicates that thepercentage of time remaining is 100% when assuming that the process wasinitiated to end in 60 seconds.

Concurrently with representation of time 2002 and representation of time2004, device 202 causes the LEDs to emit light, as shown in FIG. 20A.Like representation of time 2004, the LEDs indicate the time representedby representation of time 2002 and, thus, correspond to representationof time 2002. Here, all of the LEDs are emitting light because thepercentage of time remaining is 100% when assuming that the process wasinitiated to end in 60 seconds.

As illustrated in FIG. 20B, assume that device 202 receives anindication that the process has a remaining time of 30 seconds. Inresponse to receiving the indication, device 202 updates representationof the time 2002 and the representation of the time 2004. In FIG. 20B,representation of time 2004 includes a colored overlay that coversroughly half the area of secondary display 206. In some embodiments,representation 2004 in FIG. 20B represents that the time remaining(e.g., 30 seconds) is 50% of the original time (e.g., 60 seconds in FIG.20A) to which the process was set. In addition, device 202 also causeshalf of the LEDs (e.g., 810 a-810 l and 812 a-812 l) to cease to emitlight in accordance with the time remaining being at least 50% of theoriginal time.

As illustrated in FIG. 20C-20D, the time remaining is 15 seconds and 5seconds, respectively. As illustrated in FIG. 20C-20D, device 202continues to update primary display 204, secondary display 206, and LEDs(e.g., 810 a-810 l and 812 a-812 l) based on the remaining time (e.g.,15 second in FIG. 20C and 5 seconds in FIG. 20D) of the process.

However, contrary to FIGS. 20A-20C, device 202 causes more than theexpected portion (e.g., more than 1/12^(th)) of LEDs to emit light inFIG. 20D. Here, device 202 causes all of the LEDs to emit light toprovide an indication that the time remaining is less than apredetermined minimum time.

In some embodiments, device 202 causes one or more of the LEDs and/orsecondary display 206 to animate (e.g., pulsate, brighten) as the timeremaining moves closer to zero. In some embodiments, secondary display206 and/or the LEDs change to a different color and/or tint when apredetermined amount of time is remaining. For example, secondarydisplay 206 can change from translucent to red when a predeterminedamount of time is remaining. In some embodiments, one or more of theLEDs and/or secondary display 206 animate while one or more of the otherdisplays do not animate. In some embodiments, secondary display 206 canoverlap primary display 204, such that the animations shown in theoverlapping portion of secondary display 206 are shown in the center(e.g., an area that corresponds to an around the pupil of a user) of thedisplays of device 202.

FIGS. 21A-21D and 25 depict an exemplary technique for managing one ormore displays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 21A depicts device 202 displaying a CGR environment that includesdevice 2102 a and device 2102 b. As depicted in FIG. 21A, device 2102 aand device 2102 b are in the field-of-views of primary display 204 andsecondary display 206.

As illustrated in FIG. 21A, user interface (UI) element 2110 isdisplayed on device 2102 a. For example, UI element 2110 can be one ormore icons, files, application shortcuts, characters, etc.

With reference to FIG. 21B, device 202 detects drag-and-drop input 2150on UI element 2110. In response detecting drag-and-drop input 2150 on UIelement 2110, device 202 determines if information associated with thestatus of the drag-and-drop input can be displayed via the LEDs. Asdepicted in FIG. 21B, device 202 causes the LEDs to emit light becauseof this determination.

With reference to FIG. 21B, assume device 202 detects movement ofdrag-and-drop input 2150. In response to detecting the movement ofdrag-and-drop input 2150, device 202 displays UI element 2110 movingfrom a position on device 2102 a to a position on device 2102 b.

FIG. 21C depicts UI element 2110 at the position on device 2102 b. Asdepicted in FIG. 21C, the LEDs continue to emit light because UI element2110 remains selected.

With reference to FIG. 21C, assume device 202 detects that UI element2110 is not selected. In response to detecting that UI element 2110 isnot selected, device 202 forgoes causing the LEDs to emit light asdepicted in FIG. 21D. In some embodiments, in response to detecting thatUI element 2110 is not selected, device 202 causes one or more of theLEDs to emit light differently. For example, device 202 can cause one ormore of the LEDs to emit a light that is a different color than it waspreviously emitting in FIG. 21C while UI element 2110 was selected.

In some embodiments, causing one or more of the LEDs to emit lightdifferently indicates completion or an effect of a user action. Forexample, device 202 can cause the LEDs to emit light differently inaccordance with the rhythm of a media file that is played after a userhas dragged the media file from device 2102 a to device 2102 b.

In some embodiments, device 202 performs the techniques described above(e.g., in FIGS. 21A-21D), with respect to drag-and-drop input 2150, toindicate other user interactions and/or states of applications. Forexample, in some embodiments, device 202 performs the techniquesdescribed above when a dragging, releasing, copying, editing, clipping,or moving operation is detected. In some embodiments, device 202performs the techniques described above when device 202 receives dataconcerning a state of an application or process, such as a stateassociated with the downloading of a file, a state associated with theplaying of media, a state associated with the completion of an exerciseroutine, a state associated with the completion of periodic goals (e.g.,health goals, productivity goals), etc.

FIGS. 22A-22B and 25 depict an exemplary technique for managing one ormore displays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 22A depicts device 202 displaying a CGR environment that includesperson 2202. Person 2202 is in the field-of-views of and seen throughprimary display 204 and secondary display 206.

As depicted in FIG. 22A, person 2202 is frowning. With reference to FIG.22A, assume device 202 receives data representing the mood of person2202 that is processed by one or more algorithms, such as one or morefacial recognition algorithms. In some embodiments, the one or morealgorithms can determine that person 2202 is frowning and send data thatis representative of this determination.

In response to receiving the data representing that person 2202 isfrowning, device 202 determines that information related to the data ispermitted to be displayed via the LEDs. In some embodiments, inaccordance with this determination, device 202 causes the LEDs to emitlight in a first state. For example, FIG. 22A shows the LEDs being inthe first state (e.g., the LEDs are filled-in). In some embodiments,device 202 updates only a portion of the LEDs that are depicted solid inFIG. 22A.

In some embodiments, device 202 updates secondary display 206 with acolored overlay in a state to suggest the mood of person 22A (e.g., inaccordance with a determination that data is permitted to be displayedvia secondary display 206).

FIG. 22B depicts device 202 displaying a CGR environment that includesperson 2204. As depicted in FIG. 22B, person 2204 is smiling as opposedto person 2202 in FIG. 22A. With reference to FIG. 22B, assume device202 receives data representing the mood of person 2204 that is processedby one or more algorithms, such as one or more facial recognitionalgorithms.

In response to receiving the data representing the mood of the person2204, device 202 determines that information related to the data ispermitted to be displayed via the LEDs. In some embodiments, inaccordance with this determination, device 202 causes the LEDs to emitlight in a second state as opposed to the first state. For example, FIG.22B depicts the LEDs being in the second state by illustrating the LEDsas being hatched. Here, device 202 causes the LEDs to emit lightdifferently in the second state than in the first state because the datarepresents a different mood of the user.

In some embodiments, data representing other moods (e.g., sad, content,etc.) or other types of data can be displayed using similar techniquesto those described above in relation to FIGS. 22A-22B. For example,other types of data can include the state of financial markets, weatherforecasts, ratings (e.g., restaurant ratings, movie ratings, etc.).

FIGS. 23A-23B and 25 depict an exemplary technique for managing one ormore displays based on data associated with one or more processes inaccordance with some embodiments.

FIG. 23A depicts device 202 displaying a CGR environment that includesrepresentation 2302 of a progress indicator displayed via secondarydisplay 206. In some embodiments, representation 2302 indicates one ormore metrics that are being tracked. For example, a metric can be anumber of steps taken (or left to meet a goal), a download speed, anumber of workouts over a period of time, and/or any other metrics. Insome embodiments, representation 2302 corresponds to data received fromone or more health applications, such as a walking application, trainingapplication, strength training application, etc.

In some embodiments, device 202 causes the LEDs to emit light that isrepresentative of the progress indicator based on a determination thatinformation related to the progress indicator is permitted to bedisplayed via the LEDs. For example, as depicted in FIG. 23A, LEDs 810 aand 810 h-810 l are emitting light, where LEDs 810 a and 810 h-810 lcorrespond to the length and the position of representation 2302. LEDs810 a and 810 h-810 l indicate the level of the progress indicator.

In some embodiments, as depicted in FIG. 23A, LEDs 812 a, 812 h, 812 i,812 l emit light. LEDs 812 a, 812 h, 812 i, 812 l are on the right sideof device 202 along the length of representation 2302. LEDs 812 a, 812h, 812 i, 812 l also indicate the level of the progress indicator.Notably, device 202 LEDs 812 j and 812 k do not emit light because theyare within a certain distance from the nose of a user wearing device202.

With reference to FIG. 23B, device 202 updates representation 2302 viasecondary display 206 and the LEDs to indicate that the progress of atracked metric has been fulfilled. Notably, LEDs 812 j and 812 kcontinue not to emit light because they are within a certain distance tothe nose of a user wearing device 202.

FIGS. 24A-24D depict an exemplary technique for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments. FIGS. 24A-24D depict a scenario wheredevice 202 causes one or more of LEDs 810 a-2101 and 812 a-812 l to emitlight sequentially in a clockwise pattern around both eyes of a userwearing device 202. In some embodiments, the pattern can indicate adirection to the user and could replace or be used in addition to theexamples described above in relation to FIGS. 8A-8B, 9A-9B, 10A-10B,11A-11B, 12A-12B, 13A-13B, 14A-14B, 15, and 16.

For example, the LEDs can emit a pattern, where the LEDs turn on and offsequentially, which resembles a point moving around a circle. The circlecan be animated around one or more eyes of the user wearing device 202in a clockwise or counter-clockwise direction. In some embodiments, whendrawing the pattern around both eyes of the user, device 202 transitionsbetween the LEDs of the right and left sides of device 202, as shown inFIGS. 24B-24C, where device 202 skips a portion of the LEDs (e.g., 810 gand 810 h) in the sequence.

FIG. 25 is a flow diagram illustrating a method for managing one or moredisplays based on data associated with one or more processes inaccordance with some embodiments. In some embodiments, the method isperformed by system 100 (FIGS. 1A and 1B). In some embodiments, themethod is performed by device 202 (FIG. 2A). In some embodiments, themethod is performed by a third device or system that is different fromdevice 202 or system 100. In some embodiments, the method is performedby a combination of one or more of system 100, device 202, and the thirddevice or system.

At block 2502, information corresponding to a change in a status of aprocess is received. In some embodiments, information can be one or morenotifications, alerts, and/or one or more outputs from one or morefunctions, processes, applications, sensors, etc. In some embodiments,sensors can include one or more health sensors, heart rate sensors,pedometers, heat sensors, etc.

In some embodiments, information is computed based on one or moreenvironmental factors or data from one or more applications. Forexample, in FIGS. 20A-20D, device 202 can compute the start or end of aprocess as described above. In some embodiments, an environmental factorcan include data that corresponds to the weather, characteristics of oneor more users or devices within an environment, etc.

With reference to FIG. 25, at blocks 2504 and 2506, a firstrepresentation corresponding to the status of the process is displayedvia the secondary display in response to receiving informationcorresponding to the changed status of the process and in accordancewith a determination that first criteria are satisfied. For example, therepresentation may include one or more representations, such asrepresentation status indicator 1702 a 1 in FIG. 17B, a color overlaydisplayed across secondary display 206 in FIG. 18, indicator 1912 inFIG. 19B, representation 2004 in FIGS. 20A-20D, representation 2302 of aprogress indicator in FIGS. 23A-23B, etc.

In some embodiments, the first criteria include a criterion that is metwhen a determination is made that directional information for the objectis permitted to be displayed via the secondary display, as describedabove in relation to FIGS. 8A-8B, 9A-9B, 10A-10B, 11A-11B, 12A-12B,13A-13B, 14A-14B, 15, and 16.

With reference to FIG. 25, at blocks 2504 and 2506, a secondrepresentation corresponding to the status of the process is displayedin response to receiving information corresponding to the change instatus of the process and in accordance with a determination that secondcriteria are satisfied. For example, 810 a-810 l and 812 a-812 l inFIGS. 17A-17B, 18, 19A-19B, 20A-20D, 21A-21D, 22A-22B, 23A-23B, and24A-24D.

In some embodiments, the second criteria include a criterion that is metwhen a determination is made that the object is permitted to bedisplayed via the tertiary display as described above in relation toFIGS. 8A-8B, 9A-9B, 10A-10B, 11A-11B, 12A-12B, 13A-13B, 14A-14B, 15, and16.

In some embodiments, the second representation is different from thefirst representation. In some embodiments, the first representation is avirtual object, and the second representation is not a virtual object.

In some embodiments, one or more of the first representation and thesecond representation are associated with (e.g., corresponding to one ormore processes of the same application) a third representation that ispresented via the primary display. For example, in FIGS. 20A-20D,representation 2004 displayed via the secondary display and the LEDs(e.g., the tertiary display) are both associated with representation oftime 2002, which is displayed via primary display 204.

In some embodiments, while presenting the first representation, a secondvirtual object is concurrently displayed with the first representation.In some embodiments, the first representation highlights at least aportion of the second virtual or physical object displayed via thesecondary display. For example, in FIG. 19B, indicator 1912 highlights(or brackets) object 1910 a display via secondary display 206.

In some embodiments, one or more of the first representation and thesecond representation include a progress indicator. In some embodiments,the progress indicator updates to show a change in a metric associatedwith the device as described above in relation to representation 2302 inFIGS. 23A-23B.

In some embodiments, the first representation includes an applicationicon. In some embodiments, the process is associated with an applicationthat includes a status indicator that updates based on the process. Forexample, in FIGS. 17A-17B, status indicator 1702 a 1 is displayed inresponse to receiving data associated with a messaging application.

In some embodiments, one or more of the first representation and thesecond representation are displayed based on one or more sensors. Insome embodiments, the one or more sensors include a heart rate sensor.In some embodiments, the received information corresponds to datadetected via the heart rate sensor. In some embodiments, as a part ofdisplaying one or more of the first representation (e.g., 206 in FIG.18) and the second representation based on at least one of the one ormore sensors, an animation is displayed based data detected via heartrate sensor.

In some embodiments, the received information corresponds to an alertthat corresponds to an application. In some embodiments, one or more ofthe first representation and the second representation are presentedbased on the alert that corresponds to the application as describedabove, for example, in relation to FIG. 18.

In some embodiments, the received information corresponds to an alertthat corresponds to a calendar event. In some embodiments, one or moreof the first representation and the second representation are presentedbased on the alert that corresponds to the calendar event as describedabove, for example, in relation to FIG. 18.

In some embodiments, the received information corresponds to dataobtained via a facial recognition algorithm. In some embodiments, one ormore of the first representation and the second representation arepresented based on the data obtained via a facial recognition algorithmas described above, for example, in relation to the LEDs in FIGS. 22Aand 22B.

In some embodiments, the received information corresponds to a state ofan application. In some embodiments, one or more of the firstrepresentation and the second representation are presented based on thestate of the application. For example, in FIGS. 21A-21D, the LEDs areupdated based on whether drag-and-drop input 2150 is detected.

In some embodiments, the received information corresponds to thedetected posture of a user. In some embodiments, one or more of thefirst representation and the second representation are presented basedon a detected posture of a user wearing the device. In some embodiments,the first representation is displayed based on a detected eye positionof a user. For example, the representation can be a line that isrepresentative of the current posture or eye position of a user. In someembodiments, in response to detecting the change in posture or the eyeposition of the user, the line is updated to indicate the new (or,alternatively, the change in) posture or eye position. In someembodiments, the line is only displayed via the secondary display. Insome embodiments, the LEDs can emit light in various combinations thatare indicative of the new (or, alternatively, the change in) posture oreye position.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to providespecialized resource management of devices with additive displays (e.g.,devices with additive displays) to conserve battery life for users andto provide specialized content to users of the devices. The presentdisclosure contemplates that in some instances, this gathered data mayinclude personal information data that uniquely identifies or can beused to contact or locate a specific person. Such personal informationdata can include demographic data, location-based data, telephonenumbers, email addresses, twitter IDs, home addresses, data or recordsrelating to a user's health or level of fitness (e.g., vital signsmeasurements, medication information, exercise information), date ofbirth, or any other identifying or personal information.

The present disclosure recognizes that the use of such personalinformation data, in the present technology, can be used to the benefitof users. For example, the personal information data can be used toconserve the battery life of a user's device. Accordingly, for example,the use of such personal information data helps the system to properlymanage resources to conserve battery life for the devices. Further,other uses for personal information data that benefit the user are alsocontemplated by the present disclosure. For instance, health and fitnessdata may be used to provide insights into a user's general wellness ormay be used as positive feedback to individuals using technology topursue wellness goals.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data privacy andsecurity. Such policies should be easily accessible by users and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplates examplesin which users selectively block the use of, or access to, personalinformation data. That is, the present disclosure contemplates thathardware and/or software elements can be provided to prevent or blockaccess to such personal information data. For example, in the case ofmanaging resources for low-powered devices, the present technology canbe configured to allow users to select to “opt in” or “opt out” ofparticipation in the collection of personal information data duringregistration for services or anytime thereafter. In another example,users can select not to provide eye-tracking data, such as pupillocation, pupil dilation, and/or blink rate for specialized resourcemanagement. In yet another example, users can select to limit the lengthof time the eye-tracking data is maintained or entirely prohibit thedevelopment of a baseline eye-tracking profile. In addition to providing“opt in” and “opt out” options, the present disclosure contemplatesproviding notifications relating to the access or use of personalinformation. For instance, a user may be notified upon downloading anapp that their personal information data will be accessed and thenreminded again just before personal information data is accessed by theapp.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health-related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data at a city level rather than at an addresslevel), controlling how data is stored (e.g., aggregating data acrossusers), and/or other methods.

Therefore, although the present disclosure broadly covers the use ofpersonal information data to implement one or more various disclosedexamples, the present disclosure also contemplates that the variousexamples can also be implemented without the need for accessing suchpersonal information data. That is, the various examples of the presenttechnology are not rendered inoperable due to the lack of all or aportion of such personal information data. For example, resources oflow-powered devices can be managed and content (e.g., status updatesand/or objects) can be selected and delivered to users by inferringpreferences based on non-personal information data or a bare minimumamount of personal information, such as the content being requested bythe device associated with a user, other non-personal informationavailable to the system controlling the device, or publicly availableinformation.

What is claimed is:
 1. A head-mounted display (HMD) device, comprising:one or more image sensors; a primary display that extends across afield-of-view and has a first resolution; a secondary display that isphysically and electronically coupled to the primary display and has asecond resolution that is lower than the first resolution; one or moreprocessors; and memory storing one or more programs configured to beexecuted by the one or more processors, the one or more programsincluding instructions for: displaying a first portion of a virtualobject via the secondary display; and displaying a second portion of thevirtual object via the primary display, wherein displaying the secondportion of the virtual object via the primary display includes: inaccordance with a determination that the second portion of the virtualobject is within a predefined distance from an edge of the primarydisplay, applying a visual effect to the second portion of the virtualobject.
 2. The HMD device of claim 1, wherein the one or more programsfurther include instructions for: in accordance with a determinationthat the second portion of the virtual object is not within a predefineddistance from the edge of the primary display, forgoing to apply thevisual effect to the second portion of the virtual object.
 3. The HMDdevice of claim 1, wherein: the first portion of the virtual object hasa first visual appearance; and after applying the visual effect to thesecond portion of the virtual object, the second portion of the virtualobject has a second visual appearance that is the same as the firstvisual appearance.
 4. The HMD device of claim 1, wherein the one or moreprograms further include instructions for: concurrently displaying, viathe primary display, a third portion of the virtual object that does nothave the visual effect applied, wherein the third portion of the virtualobject does not have the visual effect applied and is less blurry thanthe second portion of the virtual object.
 5. The HMD device of claim 1,wherein the first portion of the virtual object is visuallydistinguished from the second portion of the virtual object.
 6. The HMDdevice of claim 1, wherein applying the visual effect to the secondportion of the virtual object includes: blurring the second portion ofthe virtual object.
 7. The HMD device of claim 6, wherein, after theblurring the second portion of the virtual object, the second portion ofthe virtual object includes an amount of blur that is based on an amountof blur of the first portion of the virtual object.
 8. The HMD device ofclaim 6, wherein blurring the second portion of the virtual objectincludes changing an amount of blur of the second portion of the virtualobject based on a respective distance between the virtual object and thesecondary display.
 9. The HMD device of claim 6, wherein the secondportion of the virtual object is blurred based on a characteristic of adetected, via the one or more images sensors, pupil looking through theHMD device.
 10. The HMD device of claim 1, wherein applying the secondportion of the virtual object includes: fading the second portion of thevirtual object.
 11. The HMD device of claim 10, wherein the secondportion of the virtual object is faded proportionally to a detected blurof the secondary display.
 12. The HMD device of claim 10, wherein thesecond portion of the virtual object is faded from a first color to asecond color.
 13. The HMD device of claim 12, wherein the first color isblack, and wherein the second color is white.
 14. The HMD device ofclaim 13, wherein displaying the second portion of the virtual objectincludes: rounding a corner of the second portion of the virtual object.15. The HMD device of claim 14, wherein the corner is faded.
 16. Anon-transitory computer-readable storage medium storing one or moreprograms configured to be executed by one or more processors of ahead-mounted display (HMD) device having one or more sensors, a primarydisplay that extends across a field-of-view and has a first resolution,and a secondary display that is physically and electronically coupled tothe primary display and has a second resolution that is lower than thefirst resolution, the one or more programs including instructions for:displaying a first portion of a virtual object via the secondarydisplay; and displaying a second portion of the virtual object via theprimary display, wherein displaying the second portion of the virtualobject via the primary display includes: in accordance with adetermination that the second portion of the virtual object is within apredefined distance from an edge of the primary display, applying avisual effect to the second portion of the virtual object.
 17. Thenon-transitory computer-readable storage medium of claim 16, wherein:the first portion of the virtual object has a first visual appearance;and after applying the visual effect to the second portion of thevirtual object, the second portion of the virtual object has a secondvisual appearance that is the same as the first visual appearance. 18.The non-transitory computer-readable storage medium of claim 16, whereinthe one or more programs further include instructions for: concurrentlydisplaying, via the primary display, a third portion of the virtualobject that does not have the visual effect applied, wherein the thirdportion of the virtual object does not have the visual effect appliedand is less blurry than the second portion of the virtual object. 19.The non-transitory computer-readable storage medium of claim 16, whereinapplying the visual effect to the second portion of the virtual objectincludes: blurring the second portion of the virtual object.
 20. Thenon-transitory computer-readable storage medium of claim 19, wherein,after the blurring the second portion of the virtual object, the secondportion of the virtual object includes an amount of blur that is basedon an amount of blur of the first portion of the virtual object.
 21. Thenon-transitory computer-readable storage medium of claim 19, whereinblurring the second portion of the virtual object includes changing anamount of blur of the second portion of the virtual object based on arespective distance between the virtual object and the secondarydisplay.
 22. A method for displaying a virtual object, the methodcomprising: at a system that is configured to communicate with one ormore processors, memory, one or more image sensors, and a head-mounteddisplay (HMD) device, the HMD device including a primary display thatextends across a field-of-view and has a first resolution and asecondary display that is physically and electronically coupled to theprimary display and has a second resolution that is lower than the firstresolution: displaying a first portion of a virtual object via thesecondary display; and displaying a second portion of the virtual objectvia the primary display, wherein displaying the second portion of thevirtual object via the primary display includes: in accordance with adetermination that the second portion of the virtual object is within apredefined distance from an edge of the primary display, applying avisual effect to the second portion of the virtual object.
 23. Themethod of claim 22, wherein applying the visual effect to the secondportion of the virtual object includes: blurring the second portion ofthe virtual object.
 24. The method of claim 23, wherein, after theblurring the second portion of the virtual object, the second portion ofthe virtual object includes an amount of blur that is based on an amountof blur of the first portion of the virtual object.
 25. The method ofclaim 23, wherein blurring the second portion of the virtual objectincludes changing an amount of blur of the second portion of the virtualobject based on a respective distance between the virtual object and thesecondary display.